AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

Structured Output Fails When Appending Assistant Response As JSON In Multi-Turn Conversation

When using with_structured_output in a multi-turn conversation, appending the previous assistant response serialized via model_dump_json to the message history causes the second invocation to fail with ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field.

highConfidence 85%LangchainAffected Vlangchain-Openai 1.1.14Affected Vlangchain-Core 1.3.0

Origin Analysis

The user appends the previous Pydantic response as a plain JSON string in the assistant message content. This breaks the tool-call protocol that with_structured_output relies on; the model then returns a normal assistant message without tool_calls, and the parser cannot find 'parsed' or 'refusal' in additional_kwargs.
1. Define a Pydantic model and create a structured LLM via llm.with_structured_output(MyModel).\n2. Invoke with initial system + user message; works fine.\n3. Serialize the response with MyModel.model_dump_json(response) and append as assistant message.\n4. Add a new user message and invoke again.\n5. Observe ValueError on second call.

Fixing Code Block

Edge Case Audit

Manually constructing tool_calls depends on internal tool naming and format, which may vary across different LangChain versions or model providers (e.g., NVIDIA NIM vs OpenAI). Upgrading langchain-openai or changing models could break this workaround. Additionally, if the message list is shared across threads or async tasks, in-place mutation may cause race conditions. Rollback suggestion: if this method fails, fall back to appending the serialized response as a user message instead of assistant.

Ecosystem Topology