AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

PIIMiddleware(Apply_to_output=True) Blanks Streaming Tool Call Args Due To JSON Parse Failure On Partial Chunks

When PIIMiddleware is configured with apply_to_output=True and the model streams tool call arguments in multiple chunks, the streaming redaction path treats each chunk's args fragment as complete JSON. Since each fragment is an invalid JSON string, the redactor returns an empty value, causing all tool call arguments to be erased. This leads to tool execution with no arguments and eventually a GraphRecursionError.

highConfidence 85%LangchainAffected Vlangchain-Core==1.6.4Affected Vlangchain==1.4.2

Origin Analysis

The PIIMiddleware streaming output redactor attempts to parse each streamed tool_call_chunks[].args fragment as JSON to redact PII inside tool call arguments. Real providers split a complete JSON argument string across multiple chunks (e.g., '"', '{"city"', ': "Bos', 'ton"'). Each fragment is not valid JSON, causing json.JSONDecodeError. The exception handler incorrectly returns an empty object/string instead of preserving the original fragment, so all fragments are discarded and downstream tool call accumulation assembles an empty args string.
1. Create an agent with PIIMiddleware(apply_to_output=True) and a tool that requires a parameter.\n2. Use a streaming model that emits a tool call's JSON arguments over multiple chunks with small delays between chunks.\n3. Consume the agent with stream_events(version='v3') and inspect tool_calls[].input.\n4. Observe that input is {} instead of the expected argument value, and the tool raises a validation error causing a GraphRecursionError.

Fixing Code Block

Edge Case Audit

This fix preserves arguments but does NOT guarantee PII redaction inside tool-call arguments when the complete JSON is never observed as a single chunk. For full PII protection, a follow-up change should accumulate fragments per tool call index and redact the assembled JSON after the stream ends. Additionally, if the original code shared the same dict across multiple chunks, the copy in this fix mitigates aliasing, but other parts of the streaming pipeline may still mutate shared state. Rollback: revert this patch and pin langchain-core to the previous version if integration tests reveal new regressions; ensure real streaming models with partial tool calls are tested.

Ecosystem Topology