Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever

2025-08-19

AI Summary

Tool-augmented large language models (LLMs) frequently suffer from inaccurate function calls, leading to inefficiency and increased computational costs. Existing approaches—such as fine-tuning or in-context learning with demonstrations—entail high training overheads and are vulnerable to misleading examples exhibiting behavioral inconsistency. Method: We propose the Behavior-Aligned Retriever (BAR), a retrieval-based framework grounded in contrastive learning. BAR introduces a dual-negative contrastive loss to retrieve behaviorally consistent demonstration examples from both tool-call and non-call corpora, ensuring high alignment in the underlying tool-use decision logic. Contribution/Results: BAR operates without model fine-tuning, significantly reducing spurious API invocations while preserving end-task performance. It achieves cost-effective, high-precision tool calling through behaviorally grounded example retrieval—demonstrating improved robustness, scalability, and efficiency over prior methods.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Tool-augmented large language models (LLMs) leverage external functions to extend their capabilities, but inaccurate function calls can lead to inefficiencies and increased costs.Existing methods address this challenge by fine-tuning LLMs or using demonstration-based prompting, yet they often suffer from high training overhead and fail to account for inconsistent demonstration samples, which misguide the model's invocation behavior. In this paper, we trained a behavior-aligned retriever (BAR), which provides behaviorally consistent demonstrations to help LLMs make more accurate tool-using decisions. To train the BAR, we construct a corpus including different function-calling behaviors, i.e., calling or non-calling.We use the contrastive learning framework to train the BAR with customized positive/negative pairs and a dual-negative contrastive loss, ensuring robust retrieval of behaviorally consistent examples.Experiments demonstrate that our approach significantly reduces erroneous function calls while maintaining high task performance, offering a cost-effective and efficient solution for tool-augmented LLMs.

Problem

• Reducing unnecessary API calls in tool-augmented LLMs
• Addressing inconsistent demonstration samples in function calling
• Minimizing erroneous function calls while maintaining performance

Innovation

• Behavior-aligned retriever for consistent API demonstrations
• Contrastive learning with dual-negative loss training
• Reduces erroneous function calls while maintaining performance