AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

Performance: Is_data_content_block Recomputes Block Types On Every Call, Causing Significant Overhead

The helper function _get_data_content_block_types() reconstructs its result from get_args and get_type_hints on every call, even though the result is immutable at runtime. This is called repeatedly by is_data_content_block, leading to ~23ms CPU per ChatOpenAI payload for a 204-message history, blocking the event loop in async applications.

highConfidence 95%LangChainAffected V1.6.6

Origin Analysis

The function _get_data_content_block_types() lacks caching. Each invocation rebuilds a set of type objects using get_args(DataContentBlock) and get_type_hints(DataContentBlock), which involves reflection and type resolution. Since the result is constant after import, this redundant computation wastes CPU and increases latency.
Run the provided example code with a 204-message conversation (68 iterations of Human/AI/Tool messages). Call ChatOpenAI._get_request_payload(msgs) and measure the time. The output shows 272 calls to _get_data_content_block_types per payload, taking ~23.8ms uncached. After adding lru_cache(maxsize=1) to the function, the same payload build drops to ~1.1ms.

Fixing Code Block

Edge Case Audit

Caching assumes that the type information from DataContentBlock does not change during the process lifetime. If dynamic modifications to the type hierarchy occur (e.g., monkeypatching DataContentBlock or altering type hints after import), the cached value may become stale. This scenario is highly unlikely in normal usage. To rollback, remove the decorator. lru_cache is thread-safe, so concurrent calls are safe. Use maxsize=1 to avoid unbounded memory growth.

Ecosystem Topology