Tool Learning Needs Nothing More Than a Free 8B Language Model

2026-04-19

AI Summary

This work addresses the high cost and limited flexibility of conventional tool-augmented agent training, which typically relies on annotated data or static environments built with expensive proprietary large language models. The authors propose TRUSTEE, a novel approach that, for the first time, constructs a fully simulated dynamic training environment using only a local open-source 8B-parameter language model—eliminating the need for real-world interactions, human annotations, or external resources. TRUSTEE simulates task generation, user behavior, tool responses, and trajectory evaluation entirely in silico and incorporates a multi-dimensional adaptive curriculum learning mechanism to dynamically adjust task difficulty. Experimental results demonstrate that TRUSTEE significantly outperforms baseline methods that depend on additional resources across multiple domains, highlighting the substantial potential of lightweight open-source models in tool-learning scenarios.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Reinforcement learning (RL) has become a prevalent paradigm for training tool calling agents, which typically requires online interactive environments. Existing approaches either rely on training data with ground truth annotations or require advanced commercial language models (LMs) to synthesize environments that keep fixed once created. In this work, we propose TRUSTEE, a data-free method training tool calling agents with dynamic environments fully simulated by free open-source LMs that can be as small as 8B, including task generation, user simulation, tool simulation and trajectory evaluation, paired with an adaptive curriculum learning mechanism that controls various aspects of the task difficulty dynamically during training. Our empirical results show that TRUSTEE brings consistent improvements across various domains and outperforms all the baselines which require extra external resources for training. These confirm that, with a sufficiently sophisticated design, even simulated environments with a local 8B LM as the backbone could set a strong baseline for tool learning, without expensive annotated data, realistic human interactions, executable tools or costly verifiable environments from human experts or commercial LMs. We hope our proposed paradigm could inspire future research on environment scaling with limited resources.

Problem

• tool learning
• reinforcement learning
• data-free training
• open-source language models
• simulated environments

Innovation

• tool learning
• data-free training
• open-source language models
• dynamic environment simulation
• adaptive curriculum learning