Alignment for Efficient Tool Calling of Large Language Models

2025-03-09

AI Summary

This paper addresses the prevalent issues of over-reliance and over-confidence in large language models (LLMs) during tool invocation. To recalibrate models’ awareness of their knowledge boundaries, we propose a multi-objective alignment framework. Methodologically, it introduces: (1) a novel knowledge boundary estimation technique grounded in consistency checking and absolute confidence scoring; and (2) a dynamic decision integration mechanism that jointly leverages probabilistic modeling, supervised fine-tuning, and inference-time intervention. Extensive experiments across diverse scenarios demonstrate that our approach significantly reduces redundant tool calls by 37.2% on average, while preserving task performance. Moreover, it improves response latency and lowers computational cost—achieving, for the first time, simultaneous optimization of reliability, efficiency, and cost-effectiveness in tool-augmented LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advancements in tool learning have enabled large language models (LLMs) to integrate external tools, enhancing their task performance by expanding their knowledge boundaries. However, relying on tools often introduces tradeoffs between performance, speed, and cost, with LLMs sometimes exhibiting overreliance and overconfidence in tool usage. This paper addresses the challenge of aligning LLMs with their knowledge boundaries to make more intelligent decisions about tool invocation. We propose a multi objective alignment framework that combines probabilistic knowledge boundary estimation with dynamic decision making, allowing LLMs to better assess when to invoke tools based on their confidence. Our framework includes two methods for knowledge boundary estimation, consistency based and absolute estimation, and two training strategies for integrating these estimates into the model decision making process. Experimental results on various tool invocation scenarios demonstrate the effectiveness of our framework, showing significant improvements in tool efficiency by reducing unnecessary tool usage.

Problem

• Aligning LLMs with knowledge boundaries for efficient tool usage.
• Reducing overreliance and overconfidence in tool invocation by LLMs.
• Improving tool efficiency through dynamic decision-making and boundary estimation.

Innovation

• Multi-objective alignment framework for LLMs
• Probabilistic knowledge boundary estimation
• Dynamic decision making for tool invocation