OpenAI-Compatible Responses Discard Nested Token Usage Details In UsageMetadata
langchain-openai's `_create_usage_metadata` only extracts known keys (`audio`, `cached_tokens`, `reasoning_tokens`) from `prompt_tokens_details` and `completion_tokens_details`, causing provider-specific fields like `text_tokens`, `accepted_prediction_tokens`, and `rejected_prediction_tokens` to be silently lost.
In `langchain_openai/chat_models/base.py`, `_create_usage_metadata` explicitly builds `input_token_details` and `output_token_details` by selecting only a hardcoded subset of nested token detail fields. Unknown keys returned by OpenAI-compatible providers are discarded, violating the documented extensibility of `UsageMetadata`.
1. Set up a ChatOpenAI client with base_url pointing to an OpenAI-compatible endpoint (e.g., DashScope) and stream=True.
2. Send a chat request with stream_options={'include_usage': True}.
3. Inspect the final AIMessageChunk.usage_metadata; observe that `input_token_details` only contains `cache_read` and `output_token_details` only `reasoning`, while raw usage contained `text_tokens`, `accepted_prediction_tokens`, etc.
Fixing Code Block
def _create_usage_metadata(oai_token_usage: dict, service_tier: str | None = None) -> UsageMetadata:
input_tokens = oai_token_usage.get('prompt_tokens', 0)
output_tokens = oai_token_usage.get('completion_tokens', 0)
total_tokens = oai_token_usage.get('total_tokens', input_tokens + output_tokens)
service_tier_prefix = f'{service_tier}:' if service_tier else ''
prompt_details = oai_token_usage.get('prompt_tokens_details') or {}
completion_details = oai_token_usage.get('completion_tokens_details') or {}
input_token_details: dict = {}
output_token_details: dict = {}
# Preserve standard aliases with service tier prefix
if prompt_details.get('audio_tokens') is not None:
input_token_details['audio'] = prompt_details['audio_tokens']
if prompt_details.get('cached_tokens') is not None:
input_token_details[f'{service_tier_prefix}cache_read'] = prompt_details['cached_tokens']
if completion_details.get('audio_tokens') is not None:
output_token_details['audio'] = completion_details['audio_tokens']
if completion_details.get('reasoning_tokens') is not None:
output_token_details[f'{service_tier_prefix}reasoning'] = completion_details['reasoning_tokens']
# Preserve all other provider-specific non-null fields
for key, value in prompt_details.items():
if key not in {'audio_tokens', 'cached_tokens'} and value is not None:
input_token_details[key] = value
for key, value in completion_details.items():
if key not in {'audio_tokens', 'reasoning_tokens'} and value is not None:
output_token_details[key] = value
return UsageMetadata(
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=total_tokens,
input_token_details=input_token_details,
output_token_details=output_token_details,
)
The updated function preserves the existing standard aliases (audio, cache_read, reasoning) but iterates over any remaining keys in the nested details dictionaries and includes them verbatim when non-null. This ensures provider-specific fields such as `text_tokens`, `accepted_prediction_tokens`, and `rejected_prediction_tokens` are retained in `UsageMetadata` without breaking current behavior.
Edge Case Audit
Modifying `_create_usage_metadata` affects all OpenAI-compatible responses. Providers may send arbitrary or volatile fields; storing them directly could increase metadata size and potentially introduce keys that conflict with future LangChain standard fields. Before deploying, verify that `UsageMetadata` serialization downstream (e.g., to JSON or tracing) tolerates unknown keys. If a rollback is necessary, restore the original method or pin `langchain-openai==1.1.12` until an official release includes this change.
