ModelFallbackMiddleware Strips Cache_control When Falling Back To ChatAnthropicMantle
When ModelFallbackMiddleware falls back to ChatAnthropicMantle (langchain-aws), cache_control markers are silently removed from both model_settings and message content blocks, disabling prompt caching and increasing costs. The root cause is a missing _llm_type in _ANTHROPIC_LLM_TYPES.
The _supports_anthropic_cache_control function in langchain/agents/middleware/model_fallback.py checks the model's _llm_type against a hardcoded set _ANTHROPIC_LLM_TYPES. ChatAnthropicMantle inherits from ChatAnthropic and fully supports cache_control, but its _llm_type is 'anthropic-mantle-chat', which is not included in that set. As a result, _sanitize_request_for_fallback treats it as unsupported and strips all cache_control entries.
Instantiate a ModelRequest with a ChatAnthropic model, include cache_control in a HumanMessage content block and in model_settings. Call _sanitize_request_for_fallback with fallback=ChatAnthropicMantle(...). Observe that the returned request has empty model_settings and the content block no longer contains cache_control, unlike ChatAnthropicBedrock fallback.
Fixing Code Block
# In langchain/agents/middleware/model_fallback.py
# Replace the existing _ANTHROPIC_LLM_TYPES definition with:
_ANTHROPIC_LLM_TYPES = {
"anthropic-chat",
"anthropic-bedrock-chat",
"bedrock-converse-anthropic",
"anthropic-mantle-chat", # <-- add this line
}
Adding 'anthropic-mantle-chat' to _ANTHROPIC_LLM_TYPES makes _supports_anthropic_cache_control return True for ChatAnthropicMantle, preserving cache_control during fallback sanitization. This aligns with the existing pattern for other Anthropic transports and requires no changes to the sanitization logic.
Edge Case Audit
This hotfix is specific to ChatAnthropicMantle and assumes all Mantle models support cache_control identically. If Mantle later introduces variants with different caching behavior, this could lead to invalid API requests or unexpected costs. The hardcoded set remains a maintenance hazard; consider replacing it with a capability check. To rollback, simply remove 'anthropic-mantle-chat' from the set or revert to the previous version. Also ensure tests cover the new type to avoid regression.
