Advancing SLM Tool-Use Capability using Reinforcement Learning

2025-09-03

AI Summary

Small language models (SLMs) exhibit significantly weaker tool-use capabilities than large language models (LLMs), hindering their deployment in dynamic interactive applications such as virtual assistants and automated workflows. To address this, we propose the first framework applying Group Relative Policy Optimization (GRPO)—a reinforcement learning method—to train SLMs for tool use, eliminating reliance on large-scale annotated datasets and intensive computational resources typical of supervised fine-tuning. Our approach incorporates dynamic environment feedback and jointly optimizes multi-tool selection, parameter generation, and invocation sequence planning. Experiments across multiple tool-use benchmarks demonstrate an average 23.6% improvement in accuracy, substantially narrowing the functional gap with LLMs. Moreover, inference overhead is reduced by 68%, enabling practical SLM deployment in resource-constrained settings. This work establishes a scalable, RL-driven paradigm for enhancing SLM tool competence without sacrificing efficiency.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have progressed beyond simple text creation, and tool use has become increasingly important for complex, real-world tasks. Tool use in LLMs refers to their ability to utilize external resources such as APIs, databases, or software functions to extend their functionality beyond generating text.Tools are used for tasks such as performing calculations, making API calls to retrieve the current time and date, and more. This capability enables models to fetch real-time data, execute commands, or solve problems requiring dynamic interaction, making it indispensable for applications like AI agents in virtual assistants, robotic control, or automated workflows. However, while LLMs are usually adept tool use, their vast resource requirements and computation complexity restrict their use in every use case.As a result, there is an increasing need for more compact and efficient Small Language Models (SLMs). Small language models (SLMs) struggle in tool use compared to large language models (LLMs). As soon in Table 1. SLMs are typically trained on smaller, more specific datasets, resulting in a narrower knowledge base and limited contextual understanding compared to LLMs. This research addresses these challenges by using Reinforcement Learning (RL), specifically Group Relative Policy Optimization (GRPO), to enhance tool-use proficiency in SLMs. Unlike conventional fine-tuning approaches that require heavy computation and often lack adaptability, our method provides an efficient, effective solution that significantly boosts SLM tool-use accuracy, increasing their practical utility.

Problem

• Enhancing small language models' tool-use capability
• Addressing SLMs' limited contextual understanding and knowledge
• Improving efficiency and adaptability in tool utilization

Innovation

• Reinforcement Learning enhances SLM tool-use
• Group Relative Policy Optimization boosts accuracy
• Efficient solution for small language models