🤖 AI Summary
To address scalability and training efficiency bottlenecks in deploying large language models (LLMs) as autonomous agents for multi-turn, long-horizon tasks, this paper introduces L0—a fully end-to-end reinforcement learning framework. Methodologically, L0 features: (1) NB-Agent, an agent architecture adopting a “code-as-action” REPL execution paradigm; (2) Reinforcement Learning with Verifiable Rewards (RLVR), which directly elicits problem-solving capabilities from base models without supervised fine-tuning; and (3) a lightweight sandboxed concurrent agent pool enabling high-throughput, low-cost environment interaction. Evaluated on Qwen2.5-7B-Instruct, L0 achieves substantial improvements: SimpleQA accuracy rises from 30% to 80%, and HotpotQA from 22% to 41%. The framework is open-sourced, establishing a novel paradigm for scalable, efficient training of LLM-based autonomous agents.
📝 Abstract
Training large language models (LLMs) to act as autonomous agents for multi-turn, long-horizon tasks remains significant challenges in scalability and training efficiency. To address this, we introduce L-Zero (L0), a scalable, end-to-end training pipeline for general-purpose agents. Featuring a low-cost, extensible, and sandboxed concurrent agent worker pool, L0 lowers the barrier for applying reinforcement learning in complex environments. We also introduce NB-Agent, the agent scaffold within L0, which operates in a "code-as-action" fashion via a Read-Eval-Print-Loop (REPL). We evaluate L0 on factuality question-answering benchmarks. Our experiments demonstrate that a base model can develop robust problem-solving skills using solely Reinforcement Learning with Verifiable Rewards (RLVR). On the Qwen2.5-7B-Instruct model, our method boosts accuracy on SimpleQA from 30 % to 80 % and on HotpotQA from 22 % to 41 %. We have open-sourced the entire L0 system, including our L0 series models, the NB-Agent, a complete training pipeline, and the corresponding training recipes on (https://github.com/cmriat/l0).