AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

OpenRouter 200-Response Body Errors Are Stringified As ValueError Instead Of Classified ModelError

langchain-openrouter streaming paths raise plain ValueError with the numeric error code interpolated into prose, discarding structured error classification and the is_retryable attribute. Callers cannot distinguish retryable 502/429 from non-retryable 401/403. Non-streaming invoke path also fails with a misleading no-choices ValueError because the error field is dropped during SDK parsing.

highConfidence 85%Langchain-OpenrouterAffected V<=0.2.8

Origin Analysis

In langchain_openrouter/chat_models.py, streaming error handling reads the `error` dict from a chunk and does `raise ValueError(msg)` with the code formatted into the message. The standard `langchain_core.exceptions.ModelError` subclasses (e.g., `ModelRateLimitError`, `ModelAuthError`, `ModelNotAvailableError`) are not used, so the structured numeric code and retryability are lost. The non-streaming branch cannot raise because the SDK's Pydantic completion model omits the `error` field.
See issue #40364 for full server code. Core reproduction: 1. Mock OpenRouter endpoint returns HTTP 200 with body `{"error": {"code": 502, "message": "Provider returned error"}}` (and for streaming, SSE chunks with same error). 2. Call `model.invoke("hi")` and `list(model.stream("hi"))` with `ChatOpenRouter` pointed at the mock server. 3. Observe `ValueError` exceptions, `ModelError=False`, and `is_retryable=None`.

Fixing Code Block

Edge Case Audit

This hotfix only corrects the streaming error path. The non-streaming `invoke` path remains broken: the raw error dict is stripped by the SDK's Pydantic model, so callers still get a ValueError about missing choices. A separate fix should inspect the raw HTTP response before parsing. Additionally, mapping 502 to ModelNotAvailableError assumes OpenRouter's 502 is retryable; verify against OpenRouter's error code semantics. Rollback: revert the raise replacements to the original `raise ValueError(msg)` if any regression appears after upgrading langchain-core.

Ecosystem Topology