AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

LLMToolSelectorMiddleware Ignores Conversation History, Leading To Wrong Tool Selection On Follow-Up Queries

The LLMToolSelectorMiddleware only passes the last user message to the selector model, discarding prior conversation context. This causes vague follow-up queries (e.g., 'do it') to be misinterpreted, selecting incorrect tools.

highConfidence 92%LangChainAffected V1.2.18

Origin Analysis

In _prepare_selection_request (tool_selection.py lines 211-216), the code iterates over messages in reverse to find only the last HumanMessage, and then the model invocation (lines 301-306) uses only that single message alongside the system prompt, excluding all previous messages.
1. Define two or more tools (e.g., search_documents, send_email). 2. Instantiate LLMToolSelectorMiddleware with a model and system prompt. 3. Call select() with a first message that clearly implies a tool (e.g., 'I need to send an email to my manager'). It selects correctly. 4. Call select() again with a multi-turn conversation where the last message is vague (e.g., 'do it') but prior messages establish the intended tool. 5. Observe that the middleware selects the wrong tool because it only sees the last message.

Fixing Code Block

Edge Case Audit

Using a larger message window increases token consumption and may exceed model context limits; adjust the window size (N) based on expected conversation length and model constraints. Ensure message roles are preserved as-is to avoid confusing the selector. This fix should be rolled back by reverting to the original single-message invoke if unexpected behavior occurs in other scenarios. Also note that if request.messages is very long, a slice [-5:] may still drop important earlier context; consider a more sophisticated summarization or truncation strategy.

Ecosystem Topology