🤖 AI Summary
This work addresses the limited environmental adaptability and inability of large language model (LLM) agents to reuse past experiences, which often forces them to solve tasks from scratch repeatedly. To overcome this, the authors propose the Experience Reflection Learning (ERL) framework, which, after a single task attempt, retrospectively analyzes the execution trajectory to automatically distill transferable heuristic rules. During subsequent tasks, ERL selectively retrieves relevant rules and injects them into the reasoning context to guide decision-making. This approach represents the first method to achieve cross-task knowledge abstraction through reflection and enables efficient self-improvement. Evaluated on the Gaia2 benchmark, ERL improves task success rates by 7.8% over the ReAct baseline and significantly outperforms existing experience-based learning methods, demonstrating the effectiveness of heuristic abstraction and selective retrieval.
📝 Abstract
Recent advances in large language models (LLMs) have enabled the development of autonomous agents capable of complex reasoning and multi-step problem solving. However, these agents struggle to adapt to specialized environments and do not leverage past interactions, approaching each new task from scratch regardless of their accumulated experience. We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation through experiential learning. ERL reflects on task trajectories and outcomes to generate heuristics, capturing actionable lessons that transfer across tasks. At test time, relevant heuristics are retrieved based on the current task and injected into the agent's context to guide execution. On the Gaia2 benchmark, ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability, and outperforms prior experiential learning methods. Through systematic ablations, we find that selective retrieval is essential and that heuristics provide more transferable abstractions than few-shot trajectory prompting. These results demonstrate that reflecting on single-attempt experiences to extract transferable heuristics enables effective agent self-improvement.