🤖 AI Summary
Reinforcement learning (RL) agent development has long suffered from high expertise barriers and inefficient manual iteration. This paper proposes an LLM-driven end-to-end automation framework that, for the first time, synthesizes high-performance RL agents directly from natural language task specifications and environment code. Methodologically, it introduces a dual-agent architecture that decouples MDP modeling from algorithmic optimization; an agent self-generation mechanism enabling “agents building agents”—a novel paradigm in automated RL design; and a model-context protocol integrating NLP understanding, RL formalization, adaptive training, and feedback-driven analysis. Evaluated across standard benchmarks—including MuJoCo, MetaDrive, Multi-Agent Particle Environment (MPE), and StarCraft Multi-Agent Challenge (SMAC)—the framework consistently outperforms human-designed agents, achieving substantial average performance gains, with peak improvements reaching 55%.
📝 Abstract
Reinforcement learning agent development traditionally requires extensive expertise and lengthy iterations, often resulting in high failure rates and limited accessibility. This paper introduces $Agent^2$, a novel agent-generates-agent framework that achieves fully automated RL agent design through intelligent LLM-driven generation. The system autonomously transforms natural language task descriptions and environment code into comprehensive, high-performance reinforcement learning solutions without human intervention. $Agent^2$ features a revolutionary dual-agent architecture. The Generator Agent serves as an autonomous AI designer that analyzes tasks and generates executable RL agents, while the Target Agent is the resulting automatically generated RL agent. The framework decomposes RL development into two distinct stages: MDP modeling and algorithmic optimization, enabling more targeted and effective agent generation. Built on the Model Context Protocol, $Agent^2$ provides a unified framework that standardizes intelligent agent creation across diverse environments and algorithms, while incorporating adaptive training management and intelligent feedback analysis for continuous improvement. Extensive experiments on a wide range of benchmarks, including MuJoCo, MetaDrive, MPE, and SMAC, demonstrate that $Agent^2$ consistently outperforms manually designed solutions across all tasks, achieving up to 55% performance improvement and substantial gains on average. By enabling truly end-to-end, closed-loop automation, this work establishes a new paradigm in which intelligent agents design and optimize other agents, marking a fundamental breakthrough for automated AI systems.