AI & Agent Dev Bug Sandbox logo
AI & Agent Dev Bug Sandbox
Back to Radar

ShellToolMiddleware Leaks Subprocess And Temp Directory When Agent Is Interrupted And Never Reaches After_agent

ShellToolMiddleware creates a persistent shell subprocess, reader threads, and optionally a temp directory in before_agent, but only cleans them up in after_agent. On interrupt paths (HITL, GraphBubbleUp, checkpointer pause, worker crash, or abandoned thread), after_agent is never called, and the live resources remain referenced in checkpointed state, preventing garbage collection and weakref.finalize cleanup. This leads to zombie processes, leaked temp directories, and orphaned shells accepting commands.

highConfidence 87%Langchain

Origin Analysis

The middleware stores the live _SessionResources object inside the checkpointed agent state (under 'shell_session_resources'). This strong reference keeps the object alive indefinitely, so its weakref.finalize never fires. Cleanup is only triggered by after_agent, which is bypassed on interrupt or abandonment.
1. Instantiate ShellToolMiddleware with a workspace root. 2. Call before_agent to create a shell session and obtain its PID. 3. Simulate an interrupt by deleting all references to state, resources, and middleware without calling after_agent. 4. Wait briefly and check if the subprocess is still alive using os.kill(pid, 0). 5. Observe the subprocess remains alive and the temp directory (if workspace_root was None) is not removed.

Fixing Code Block

Edge Case Audit

1. TTL value must be tuned: too short risks killing a live session during a legitimate long-running agent or a delayed resume; too long prolongs the leak. Consider making TTL configurable. 2. Multi-threaded concurrency: the cache relies on unique UUID keys; ensure no collisions if the same state is processed concurrently. 3. If the process crashes hard (SIGKILL), the atexit handler may not run, and the cache is lost, leaving orphaned subprocesses. Combine with process-group kill in the subprocess creation to mitigate. 4. Rolling back this fix restores the original behavior; before rollback, manually clean up any resources still in the cache (e.g., call ShellResourceManager.cleanup for all keys) to avoid leaving zombies. 5. Existing checkpoints that still contain 'shell_session_resources' objects will not be compatible unless migration is handled; provide a migration step to extract and clean them.

Ecosystem Topology