AI & Agent Dev Bug Sandbox Back to Radar

Unsafe Release Ordering: Langchain-Tests May Precede Integration Package Updates For Tool_call_streaming

The issue tracks the need to hold integration package releases until langchain-tests is released, because the stricter standard tests depend on updated provider profiles. Releasing in the wrong order can cause users to encounter incompatible standard tests.

mediumConfidence 85%Langchain Affected Vlangchain-Core==1.4.5 Affected Vlangchain-Model-Profiles==0.0.6

Origin Analysis

The release pipeline lacks explicit dependency ordering between integration packages and langchain-tests. The standard test feature is profile-gated but for opted-in providers it requires refreshed runtime profile data from partner packages. If langchain-tests is released first, users may install it while provider profiles still lack the new tool_call_streaming flag, leading to test failures or false positives.

1. Have langchain-core==1.4.5 and langchain-model-profiles==0.0.6 released. 2. Do not release the updated integration packages (langchain-openai, langchain-anthropic, etc.). 3. Release langchain-tests with the stricter tool_call_streaming standard test. 4. A user upgrades langchain-tests and runs standard tests against an opt-in provider (e.g., OpenAI). 5. The standard test expects tool_call_streaming in the model profile, but the provider package still has old profile data without that flag, causing test failure or unexpected behavior.

Fixing Code Block

name: Coordinated Release

on:
  workflow_dispatch:

jobs:
  release-integrations:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        package:
          - langchain-openai
          - langchain-anthropic
          - langchain-fireworks
          - langchain-huggingface
          - langchain-openrouter
          - langchain-deepseek
          - langchain-xai
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install build tools
        run: pip install build twine
      - name: Build and publish ${{ matrix.package }}
        env:
          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
        run: |
          cd ${{ matrix.package }}
          python -m build
          twine upload dist/*

  release-tests:
    needs: release-integrations
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install build tools
        run: pip install build twine
      - name: Build and publish langchain-tests
        env:
          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
        run: |
          cd langchain-tests
          python -m build
          twine upload dist/*

This workflow enforces the required release order by using GitHub Actions job dependencies. The `release-tests` job will only run after all integration package releases in `release-integrations` have completed successfully, ensuring that updated provider profiles are available before the stricter standard tests are published.

Edge Case Audit

This fix relies on the success of all matrix jobs; if any integration package fails to publish, the entire `release-integrations` job fails and `release-tests` will not run, which is good. However, partial releases may still occur (some packages published before failure), leading to an inconsistent state where some provider profiles are updated and others are not. In such a case, rollback is difficult because PyPI does not support removal of published packages. Recommendation: use a staging index or release candidates, and verify all packages before final publication. Also, if a rollback is needed after langchain-tests is released, you must yank the langchain-tests release and potentially re-release a previous version to avoid users picking up incompatible tests.