ChatFireworks Does Not Surface LangSmith Gateway Metadata On Successful Responses
Fireworks integration lacks parsing of the X-LangSmith-Gateway-Metadata header on successful responses when using LANGSMITH_GATEWAY, causing missing gateway info and incorrect model/provider in traces.
ChatFireworks calls chat.completions.create directly, discarding HTTP headers. Unlike ChatOpenAI and ChatAnthropic, it does not use with_raw_response to capture the X-LangSmith-Gateway-Metadata header, so the tracer cannot promote it to ls_gateway_info.
1. Set LANGSMITH_GATEWAY=true and use ChatFireworks with a valid lsv2_ API key. 2. Run a completion request. 3. Observe that generation_info lacks GATEWAY_METADATA_RESPONSE_KEY, and trace shows requested model/provider instead of gateway-resolved values.
Fixing Code Block
from langchain_core.utils._gateway import GATEWAY_METADATA_RESPONSE_KEY, _parse_gateway_metadata
# Inside ChatFireworks class, add:
@property
def _uses_gateway(self) -> bool:
return bool(self.fireworks_api_key and self.fireworks_api_key.startswith("lsv2_"))
def _add_gateway_metadata(self, generation_info: dict, headers: Optional[Mapping[str, str]]) -> None:
if not headers:
return
try:
metadata = _parse_gateway_metadata(headers)
if metadata:
generation_info[GATEWAY_METADATA_RESPONSE_KEY] = metadata
except Exception:
pass
def _completion_with_retry(self, **kwargs: Any) -> Tuple[Any, Optional[Mapping[str, str]]]:
if self._uses_gateway:
raw_response = self.client.with_raw_response.create(**kwargs)
response = raw_response.parse()
return response, raw_response.headers
return self.client.create(**kwargs), None
async def _acompletion_with_retry(self, **kwargs: Any) -> Tuple[Any, Optional[Mapping[str, str]]]:
if self._uses_gateway:
raw_response = await self.async_client.with_raw_response.create(**kwargs)
response = raw_response.parse()
return response, raw_response.headers
return await self.async_client.create(**kwargs), None
# In _generate, adapt call to _completion_with_retry:
response, headers = self._completion_with_retry(**params)
chat_result = self._create_chat_result(response)
if self._uses_gateway:
for generation in chat_result.generations:
if isinstance(generation, ChatGeneration):
self._add_gateway_metadata(generation.generation_info or {}, headers)
# In _stream, adapt similarly:
stream, headers = self._completion_with_retry(**params, stream=True)
first_chunk = True
for chunk in stream:
if first_chunk and self._uses_gateway and headers:
chunk_generation_info = getattr(chunk, "generation_info", None) or {}
self._add_gateway_metadata(chunk_generation_info, headers)
if hasattr(chunk, "generation_info"):
chunk.generation_info = chunk_generation_info
first_chunk = False
yield self._convert_chunk_to_generation_chunk(chunk, run_manager)
Adds a _uses_gateway property checking for lsv2_ API key prefix. When true, calls client.with_raw_response.create to retain HTTP headers, parses gateway metadata via shared helper, and injects it into generation_info for both sync/async and streaming/non-streaming paths. Non-gateway usage remains unchanged.
Edge Case Audit
Only activates for gateway API keys (lsv2_ prefix), so custom clients and non-gateway usage are unaffected. Edge case: raw 5xx HTTPStatusError after retries lacks response headers; not addressed here. Ensure langchain-core version includes _parse_gateway_metadata; fallback to no-op on import failure. Backward compatible: if headers missing, generation_info unchanged. For rollback, revert these specific code changes; no database or state changes involved.
