AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

ChatGroq._Combine_llm_outputs Mutates Per-Generation Token_usage In Place And Mishandles None Values (Silent Clobber / TypeError)

The _combine_llm_outputs method in ChatGroq has three defects: (1) it mutates the caller's per-generation token_usage dicts in place due to storing references, (2) it silently overwrites accumulated numeric values with None, and (3) it crashes with TypeError when None is encountered first and later a numeric value is added. This corrupts callback/trace data and breaks batched generation.

highConfidence 95%LangchainAffected V1.1.3

Origin Analysis

The merge loop stores nested usage dicts by reference (`overall_token_usage[k] = v`) and then uses in-place `+=` on nested values. The None-guard only checks the incoming value (`else: overall_token_usage[k] = v`), so a later None overwrites an existing number, while a None stored first then combined with a number causes `+=` on NoneType.
1. Create ChatGroq(llm='llama-3.3-70b-versatile'). 2. Call _combine_llm_outputs with two outputs having nested token_usage dicts containing 'input_tokens_details'. 3. Observe that the first output's nested dict is mutated and shared with the combined result. 4. Call with two outputs where a token_usage value is 0.5 in first and None in second; observe 0.5 is silently lost. 5. Call with first value None and second value 0.5; observe TypeError: unsupported operand type(s) for +=: 'NoneType' and 'float'.

Fixing Code Block

Edge Case Audit

The provided code assumes usage dict values are either None, dict, or numeric (int/float). If non-numeric non-None values (e.g., strings) are ever introduced by the Groq API, `+=` may still throw TypeError; consider adding explicit type checking if such cases emerge. This change is localized and should be tested against existing Groq response schemas. To rollback, replace the method with the original code from langchain-groq 1.1.3 or pin the package version. Concurrency is not directly impacted because _combine_llm_outputs operates on per-call data, but ensure no shared mutable state is introduced elsewhere.

Ecosystem Topology