AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

EvaluationResult.Feedback_config Silently Drops Unknown Or Partial Dict Fields Due To Coercion To FeedbackConfig

When a dict is passed to EvaluationResult.feedback_config, the dataclass __post_init__ coerces it to FeedbackConfig, which ignores unknown fields. This causes silent data loss and contradicts the Union[FeedbackConfig, dict] type hint.

highConfidence 95%Langchain_coreAffected V0.3.65

Origin Analysis

In langchain_core.tracers.evaluation.EvaluationResult.__post_init__, any dict assigned to feedback_config is passed to FeedbackConfig(**dict). FeedbackConfig is a Pydantic model with extra='ignore', so keys not matching model fields are silently discarded. Because dict is allowed in the type hint but coerced, arbitrary dict configurations cannot be stored.
```python\nfrom langchain_core.tracers.evaluation import EvaluationResult\n\nresult = EvaluationResult(\n key=\"sentiment\",\n value=\"positive\",\n feedback_config={\"threshold\": 1.0}\n)\nprint(result.feedback_config) # Output: FeedbackConfig() or missing threshold key\n```

Fixing Code Block

Edge Case Audit

Some downstream code may currently assume feedback_config is an instance of FeedbackConfig and access model-specific attributes (e.g., result.feedback_config.type). With this change, such code will raise AttributeError if a dict was passed. Defensive code should check isinstance(result.feedback_config, FeedbackConfig) before attribute access. Rollback suggestion: if strict validation is required, replace the silent coercion with an explicit validator that raises on unknown fields instead of dropping them, rather than reverting to silent loss.

Ecosystem Topology