Steve-Evolving: Open-World Embodied Self-Evolution via Fine-Grained Diagnosis and Dual-Track Knowledge Distillation

2026-03-13

AI Summary

This work addresses the performance bottleneck of open-world embodied agents in long-horizon tasks, which stems from inadequate mechanisms for organizing and evolving experiential knowledge. To overcome this limitation, the authors propose a non-parametric self-evolution framework that enables structured experience anchoring, distillation of skills and executable safeguards, and online updating of LLM-based planners through fine-grained execution diagnostics and a dual-track knowledge distillation loop. Innovatively, a three-stage closed-loop mechanism integrates structured experience tuples, multi-dimensional indexing, and compositional diagnostic signals to support continual learning and decision optimization without requiring model parameter updates. Evaluated on the Minecraft MCU long-horizon task suite, the method significantly outperforms static retrieval baselines, demonstrating its effectiveness and scalability in complex, open-ended environments.

Technology Category

• Multiagent Systems: Multiagent Learning
• Intelligent Robots: Embodied AI
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Open-world embodied agents must solve long-horizon tasks where the main bottleneck is not single-step planning quality but how interaction experience is organized and evolved. To this end, we present Steve-Evolving, a non-parametric self-evolving framework that tightly couples fine-grained execution diagnosis with dual-track knowledge distillation in a closed loop. The method follows three phases: Experience Anchoring, Experience Distillation, and Knowledge-Driven Closed-Loop Control. In detail, Experience Anchoring solidifies each subgoal attempt into a structured experience tuple with a fixed schema (pre-state, action, diagnosis-result, and post-state) and organizes it in a three-tier experience space with multi-dimensional indices (e.g., condition signatures, spatial hashing, and semantic tags) plus rolling summarization for efficient and auditable recall. To ensure sufficient information density for attribution, the execution layer provides compositional diagnosis signals beyond binary outcomes, including state-difference summaries, enumerated failure causes, continuous indicators, and stagnation/loop detection. Moreover, successful trajectories of Experience Distillation are generalized into reusable skills with explicit preconditions and verification criteria, while failures are distilled into executable guardrails that capture root causes and forbid risky operations at both subgoal and task granularities. Besides, Knowledge-Driven Closed-Loop Control retrieved skills and guardrails are injected into an LLM planner, and diagnosis-triggered local replanning updates the active constraints online, forming a continual evolution process without any model parameter updates. Experiments on the long-horizon suite of Minecraft MCU demonstrate consistent improvements over static-retrieval baselines.

Problem

• open-world embodied agents
• long-horizon tasks
• experience organization
• self-evolution
• execution diagnosis

Innovation

• fine-grained diagnosis
• dual-track knowledge distillation
• non-parametric self-evolution
• experience anchoring
• knowledge-driven closed-loop control