Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning

2026-06-23

AI Summary

This work addresses the challenge that large language model agents, when learning from experience, are prone to confirmation bias—mistaking internally consistent yet incorrect behaviors as successful, thereby accumulating errors. To mitigate this, the paper introduces the EDV framework, which restructures experiential learning from a single-agent reflective loop into a multi-agent collaborative mechanism. EDV decouples learning into three stages—execution, distillation, and verification—and integrates heterogeneous agents for parallel exploration, third-party comparative distillation, consensus-based validation, and a hybrid shared/private memory strategy to effectively filter noisy or erroneous experiences. Evaluated on three long-horizon task benchmarks—tau2-bench, Mind2Web, and MMTB—EDV significantly outperforms strong baselines, demonstrating that constructing reliable experiences is crucial for agent self-improvement.

Abstract

Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction. However, existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self-Confirmation Trap: wrong-but-self-consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute-Distill-Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third-party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor-centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self-reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long-horizon benchmarks: tau2-bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self-evolution. Our code is available at https://github.com/shidingz/EDV.

Problem

• Self-Confirmation Trap
• experience learning
• LLM agents
• memory bias
• trajectory validation

Innovation

• Execute-Distill-Verify
• Self-Confirmation Trap
• Agentic Experience Learning
• Heterogeneous Agents
• Consensus Verification