AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

InMemoryVectorStore Silently Drops Documents When Embedding Returns Mismatched Vector Count

`InMemoryVectorStore.add_documents` and `aadd_documents` silently drop documents when the embedding model returns fewer vectors than the number of input documents. The root cause is the use of `zip(..., strict=False)` in the underlying `add_texts`/`aadd_texts` methods. A `ValueError` should be raised immediately to prevent silent data loss.

highConfidence 95%Langchain-CoreAffected V1.3.0a1

Origin Analysis

The `add_texts` and `aadd_texts` methods of `InMemoryVectorStore` use `zip(texts, vectors)` without `strict=True`. Python's `zip` defaults to `strict=False`, which silently truncates to the shortest iterable. When `Embeddings.embed_documents` returns fewer vectors than the number of input texts (e.g., because of an off-by-one bug in a custom embedding implementation or a batched provider truncation), the extra documents are never stored, and the caller receives fewer IDs with no error.
1. Define a custom `Embeddings` subclass that returns one fewer vector than the number of input texts in `embed_documents`. 2. Instantiate `InMemoryVectorStore` with that embedding. 3. Call `store.add_documents([Document('first'), Document('second'), Document('third')])`. 4. Observe that it returns only two IDs and no exception is raised, while the third document is silently dropped.

Fixing Code Block

Edge Case Audit

This fix requires Python 3.10 or later because `zip(..., strict=True)` was introduced in Python 3.10. If the project still supports Python 3.9 or earlier, the `strict=True` argument will raise `TypeError`; in that case, use only the explicit length check and omit `strict=True`. The change may cause previously silent embedding truncations to surface as runtime `ValueError`s, which is intended but could break existing deployments that accidentally relied on the old behavior. InMemoryVectorStore is not thread-safe for concurrent writes, and this fix does not change that; concurrent callers may still lose data if the same store is modified without external synchronization. Rollback is possible by removing the length checks and `strict=True`, but doing so will re-introduce silent data loss.

Ecosystem Topology