Building Self-Evolving Agents via Experience-Driven Lifelong Learning: A Framework and Benchmark

2025-08-26

AI Summary

This work addresses the challenge of continual learning and self-evolution for intelligent agents in open-ended environments. We propose the Experience-driven Lifelong Learning (ELL) framework, which transcends static task optimization toward general artificial intelligence. ELL integrates dynamic environmental interaction, long-term memory, experience abstraction, skill validation, and knowledge internalization, augmented by context engineering to enhance large language models’ stability and adaptability in lifelong learning. Concurrently, we introduce StuLife—a novel student life-cycle simulation benchmark—designed to systematically evaluate autonomous decision-making, cross-task skill transfer, and long-term memory retention. Experiments demonstrate that ELL significantly improves large models’ continual learning capability and generalization performance in open environments. Our framework establishes a new paradigm for developing agentic systems endowed with self-motivation and evolutionary capacity.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

As AI advances toward general intelligence, the focus is shifting from systems optimized for static tasks to creating open-ended agents that learn continuously. In this paper, we introduce Experience-driven Lifelong Learning (ELL), a framework for building self-evolving agents capable of continuous growth through real-world interaction. The framework is built on four core principles: (1) Experience Exploration: Agents learn through continuous, self-motivated interaction with dynamic environments, navigating interdependent tasks and generating rich experiential trajectories. (2) Long-term Memory: Agents preserve and structure historical knowledge, including personal experiences, domain expertise, and commonsense reasoning, into a persistent memory system. (3) Skill Learning: Agents autonomously improve by abstracting recurring patterns from experience into reusable skills, which are actively refined and validated for application in new tasks. (4) Knowledge Internalization: Agents internalize explicit and discrete experiences into implicit and intuitive capabilities as "second nature". We also introduce StuLife, a benchmark dataset for ELL that simulates a student's holistic college journey, from enrollment to academic and personal development, across three core phases and ten detailed sub-scenarios. StuLife is designed around three key paradigm shifts: From Passive to Proactive, From Context to Memory, and From Imitation to Learning. In this dynamic environment, agents must acquire and distill practical skills and maintain persistent memory to make decisions based on evolving state variables. StuLife provides a comprehensive platform for evaluating lifelong learning capabilities, including memory retention, skill transfer, and self-motivated behavior. Beyond evaluating SOTA LLMs on the StuLife benchmark, we also explore the role of context engineering in advancing AGI.

Problem

• Developing self-evolving agents for continuous learning through real-world interaction
• Creating a benchmark to evaluate lifelong learning capabilities in dynamic environments
• Advancing AGI by integrating experience exploration and long-term memory systems

Innovation

• Experience-driven lifelong learning framework
• Persistent memory system integration
• Autonomous skill abstraction refinement