✦ Continue with Google
AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

LangSmith Hub PullPromptCommit Manifest Omits Custom Base_url And Uses Wrong Secret Id For OpenAI-Compatible Models

When a prompt is configured in LangSmith UI with an OpenAI-compatible provider, custom base_url (e.g., OpenRouter) and a custom API key env name (e.g., OPENROUTER_API_KEY), the Playground works correctly but Client.pullPromptCommit returns a manifest for the same commit that omits base_url and resolves the secret to OPENAI_API_KEY, causing 401/wrong endpoint in application code.

highConfidence 78%LangsmithAffected V0.5.23

Origin Analysis

The LangSmith Hub backend serialization for OpenAI-compatible model configs does not preserve non-default endpoint (base_url) and custom secret identifier fields when hydrating the ChatOpenAI constructor in the manifest. The UI stores and displays the custom configuration, but the API manifest only includes standard OpenAI fields (model, use_responses_api, openai_api_key), indicating a desync between UI-stored config and the serialized Hub manifest.
1. Create a prompt in LangSmith UI with Provider: OpenAI-compatible, Base URL: https://openrouter.ai/api/v1, API key env name: OPENROUTER_API_KEY, Model: google/gemini-3-flash-preview. 2. Save and create a new commit. 3. In application code, call Client.pullPromptCommit with includeModel: true and skipCache: true. 4. Inspect the ChatOpenAI kwargs in the returned manifest; observe that base_url is missing and openai_api_key secret id is ['OPENAI_API_KEY'] instead of ['OPENROUTER_API_KEY'].

Fixing Code Block

Edge Case Audit

This patch is a temporary workaround until the LangSmith backend correctly serializes base_url and custom secret ids. Do not apply it globally to all prompts if some use standard OpenAI; guard by checking prompt metadata or known identifiers. The function deep-copies the manifest to avoid mutating shared state, but concurrent code should still avoid repeatedly patching an already-correct manifest (guard for existing base_url). When the upstream fix lands, this patch could duplicate base_url or override the correct secret if the manifest already contains the expected fields; add a check before patching. Rollback is simply to not call the patch function or to revert to the original manifest object. No destructive operations are performed, but ensure the base_url and secret_ref values are correct for the target prompt to avoid unintended routing or credential use.

Ecosystem Topology