AI & Agent Dev Bug Sandbox Back to Radar

SummarizationMiddleware Fails To Identify Claude On Bedrock, Causing Late Summarization And Potential Context Overflow

The SummarizationMiddleware uses a hardcoded check on model._llm_type to apply Claude-specific token calibration. For ChatBedrockConverse (and ChatBedrock), _llm_type is 'amazon_bedrock_converse_chat', which does not match the expected 'anthropic-chat' prefix. As a result, it falls back to the generic chars_per_token=4.0 instead of Claude's calibrated 3.3, underestimating token counts by ~20%. This leads to summarization triggering too late and can cause model calls to fail with 'Input is too long for requested model'.

highConfidence 95%LangChain Affected V<=1.4.1

Origin Analysis

In langchain.agents.middleware.SummarizationMiddleware._get_approximate_token_counter, the logic checks if self.model._llm_type.startswith("anthropic-chat") to apply chars_per_token=3.3. Claude models served via Amazon Bedrock use _llm_type='amazon_bedrock_converse_chat' (or similar), failing this check and defaulting to 4.0. The design flaw is coupling token calibration to a specific provider-specific _llm_type instead of detecting the underlying model family (Claude).

Run the following Python snippet with langchain, langchain-anthropic, and langchain-aws installed:\n```python\nfrom langchain.agents.middleware import SummarizationMiddleware\nfrom langchain_anthropic import ChatAnthropic\nfrom langchain_aws import ChatBedrockConverse\n\nmodels = [\n ChatAnthropic(model=\"claude-sonnet-4-5\", api_key=\"unused\"),\n ChatBedrockConverse(\n model=\"us.anthropic.claude-sonnet-4-5-20250929-v1:0\", region_name=\"us-east-1\"\n ),\n]\nfor model in models:\n mw = SummarizationMiddleware(model=model, trigger=(\"tokens\", 170_000))\n print(f\"{type(model).__name__:22} _llm_type={model._llm_type!r:34} chars_per_token={mw.token_counter.keywords.get('chars_per_token', 4.0)}\")\n```\nExpected output shows ChatAnthropic gets 3.3 but ChatBedrockConverse gets 4.0.

Fixing Code Block

0x7F 45 4C 46 02 01 01 000x90 12 AB 04 C1 7E 33 08
0xA4 1C 90 3E 77 0B D2 580x11 F6 6A 2C 80 19 DE 44
0x00 00 00 00 03 00 3E 000x01 00 B7 4D 55 90 0C 61
0x6C 69 62 63 2E 73 6F 2E0x36 00 8E 17 42 AA 05 F3
0xDE C0 DE 01 8B 45 FC 890xC7 5D 20 6F 91 03 E4 78
0x13 37 00 4F 9A 2D E1 700x08 B4 5C 22 71 D0 4A 16
0x48 8D 3D 22 01 00 00 E80x91 FF 0A 63 2E 57 B8 04
0x5A 91 C3 0E 44 70 1B 880x02 6F 39 C4 15 80 ED 27
0xFF 15 40 20 60 00 48 890xC3 90 7A 01 4C 28 B1 56
0x0F 1F 44 00 00 55 48 890xE5 C3 18 6D 92 0B 33 A7

Edge Case Audit

The proposed fix relies on the model's _get_ls_params() method returning a dictionary with 'ls_model_name'. While this is generally stable in current LangChain versions, future changes to the internal LS params structure could break detection. Additionally, a model name containing 'claude' is assumed to be Anthropic Claude; this is safe today but could theoretically cause false positives if another vendor uses that string. No concurrency or multithreading issues are introduced as the method is stateless. To roll back, simply restore the original _llm_type-only check. Recommend adding unit tests for ChatAnthropic, ChatBedrock, ChatBedrockConverse, and a generic model to ensure correct calibration.