AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

Streamed Refusals Are Dropped, So Structured Output Raises A Generic `ValueError` Instead Of `OpenAIRefusalError`

When using streaming with structured outputs on OpenAI Chat Completions, a model refusal is lost because `_convert_delta_to_message_chunk` does not read the `refusal` field from the delta. Consequently, `_oai_structured_outputs_parser` cannot find a refusal and raises a generic `ValueError` instead of `OpenAIRefusalError`. The non-streaming path already records refusals, creating an inconsistency. A second, narrower gap exists for OpenAI-compatible endpoints returning plain dicts via `base_url`.

highConfidence 85%Langchain-Openai

Origin Analysis

`_convert_delta_to_message_chunk` and `_convert_dict_to_message` do not preserve the `refusal` field from OpenAI deltas/dicts into `additional_kwargs`. The structured output parser checks `additional_kwargs["refusal"]`, but because it is missing, the parser falls through to the generic `ValueError` path.
1. Call a streaming Chat Completions request with structured output.\n2. Have the model refuse the request.\n3. Observe that `_oai_structured_outputs_parser` raises `ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field` and the refusal text is not available. The provided issue script reproduces this using `_convert_delta_to_message_chunk` with deltas containing `refusal`.

Fixing Code Block

Edge Case Audit

This change broadens the surface where `refusal` is recorded: any OpenAI-compatible endpoint that returns a `refusal` key in a dict will now expose it in `additional_kwargs`, which could affect downstream consumers that do not expect the key. If a custom schema or endpoint uses `refusal` with non-string values or for purposes other than safety refusals, chunk merging may produce unexpected results. Rolling back is straightforward: revert to the previous version of the two functions. Testing should cover both streaming and non-streaming OpenAI responses, dict-based OpenAI-compatible endpoints, and multi-chunk refusals to ensure no regressions in message merging.

Ecosystem Topology