Self-evolved Imitation Learning in Simulated World

2025-09-23

AI Summary

To address the challenges of scarce expert demonstrations and poor cross-task generalization in few-shot imitation learning, this paper proposes Self-Evolving Imitation Learning (SEIL). SEIL enables agents to autonomously generate successful trajectories via simulator-based exploration, iteratively improving policy capability. It introduces a dual-level augmentation mechanism—EMA-based model collaboration and environmental initial-state perturbation—to enhance trajectory diversity, and incorporates a lightweight selector to automatically identify high-quality, complementary trajectories for training. By significantly reducing reliance on human-annotated demonstrations, SEIL achieves new state-of-the-art performance on the LIBERO benchmark for few-shot imitation learning. Notably, it attains superior cross-task generalization using only a minimal number of expert demonstrations. The implementation is publicly available.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Sampling/Simulation-based Search
• Multiagent Systems: Multiagent Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Imitation learning has been a trend recently, yet training a generalist agent across multiple tasks still requires large-scale expert demonstrations, which are costly and labor-intensive to collect. To address the challenge of limited supervision, we propose Self-Evolved Imitation Learning (SEIL), a framework that progressively improves a few-shot model through simulator interactions. The model first attempts tasksin the simulator, from which successful trajectories are collected as new demonstrations for iterative refinement. To enhance the diversity of these demonstrations, SEIL employs dual-level augmentation: (i) Model-level, using an Exponential Moving Average (EMA) model to collaborate with the primary model, and (ii) Environment-level, introducing slight variations in initial object positions. We further introduce a lightweight selector that filters complementary and informative trajectories from the generated pool to ensure demonstration quality. These curated samples enable the model to achieve competitive performance with far fewer training examples. Extensive experiments on the LIBERO benchmark show that SEIL achieves a new state-of-the-art performance in few-shot imitation learning scenarios. Code is available at https://github.com/Jasper-aaa/SEIL.git.

Problem

• Addresses limited expert demonstrations in multi-task imitation learning
• Proposes self-evolved framework using simulator interactions for iterative refinement
• Enhances demonstration diversity through dual-level augmentation strategies

Innovation

• Progressive few-shot model improvement through simulator interactions
• Dual-level augmentation with EMA model and environment variations
• Lightweight selector filters complementary trajectories for quality