AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use

2026-04-23

AI Summary

This work addresses the pressing industrial demand for cost-effective, low-latency small language agents capable of multi-step reasoning and tool use by proposing a multi-turn reinforcement learning framework integrated with a dual data flywheel mechanism. The framework features a reasoning flywheel that automatically generates increasingly challenging tasks through error analysis and an agent flywheel that extends linear workflows into multi-branch behavior trees to better approximate real-world decision complexity. It further incorporates synthetic data generation, behavior tree modeling, and knowledge distillation tailored for small models. Experimental results demonstrate that the proposed approach achieves strong performance across multiple public agent benchmarks and substantially narrows the performance gap between small and large models on industrial-scale search and data analysis tasks.

Technology Category

• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Modern industrial applications increasingly demand language models that act as agents, capable of multi-step reasoning and tool use in real-world settings. These tasks are typically performed under strict cost and latency constraints, making small agentic models highly desirable. In this paper, we introduce the AgenticQwen family of models, trained via multi-round reinforcement learning (RL) on synthetic data and a limited amount of open-source data. Our training framework combines reasoning RL and agentic RL with dual data flywheels that automatically generate increasingly challenging tasks. The reasoning flywheel increases task difficulty by learning from errors, while the agentic flywheel expands linear workflows into multi-branch behavior trees that better reflect the decision complexity of real-world applications. We validate AgenticQwen on public benchmarks and in an industrial agent system. The models achieve strong performance on multiple agentic benchmarks, and in our industrial agent system, close the gap with much larger models on search and data analysis tasks. Model checkpoints and part of the synthetic data: https://huggingface.co/collections/alibaba-pai/agenticqwen. Data synthesis and RL training code: https://github.com/haruhi-sudo/data_synth_and_rl. The data synthesis pipeline is also integrated into EasyDistill: https://github.com/modelscope/easydistill.

Problem

• agentic language models
• tool use
• industrial-scale
• small models
• multi-step reasoning

Innovation

• dual data flywheels
• agentic language models
• reinforcement learning
• behavior trees
• synthetic data generation