AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

ModelRouterMiddleware Overrides ModelFallbackMiddleware Fallback Models

When ModelRouterMiddleware is composed inside ModelFallbackMiddleware, the router unconditionally replaces the fallback model with the originally selected routed model, causing the fallback model to be skipped and the same failing model to be called twice.

highConfidence 85%LangchainAffected V0.0.1a3

Origin Analysis

ModelRouterMiddleware.wrap_model_call (and awrap_model_call) always invokes handler(request.override(model=self.models[answer.choice])) without checking whether the incoming request already carries a model explicitly set by an outer middleware. When ModelFallbackMiddleware creates a replacement request with its fallback model, the router treats it as a fresh routing decision and overrides the fallback model with the previously selected failing model.
1. Create a routed model that always raises RuntimeError and a fallback model that returns a valid message.\n2. Wrap TypeSafeClassifier with a fake returning a chosen route.\n3. Instantiate ModelRouterMiddleware with the routed model.\n4. Create an agent using middleware=[ModelFallbackMiddleware(fallback_model), router].\n5. Invoke the agent with any input.\n6. Observe that the routed model is attempted twice and the final exception is raised, instead of returning the fallback response.

Fixing Code Block

Edge Case Audit

The mutable _original_model instance attribute is not thread-safe or async-safe if a single ModelRouterMiddleware instance is shared across concurrent requests. In concurrent environments, the first request may set _original_model incorrectly, causing subsequent requests to skip routing or route incorrectly. Defensive suggestion: use a contextvars.ContextVar or store the original model per request via request metadata instead of using instance state. Additionally, if another middleware modifies the request model before the router's first invocation, the recorded 'original' model will not be the actual base model, leading to incorrect routing decisions. Rollback: revert this change or temporarily reorder middleware to router outside fallback as a workaround.

Ecosystem Topology