Uncertainty Quantification for LLM Function-Calling

2026-04-24

AI Summary

Large language models (LLMs) may execute irreversible operations—such as financial transfers or data deletion—due to high-confidence erroneous function calls, necessitating reliable quantification of their invocation uncertainty. This work presents the first systematic evaluation of uncertainty quantification methods in the context of function calling, revealing limited gains from existing multi-sample strategies. To address this, the study introduces two targeted improvements: an abstract syntax tree–based clustering approach to optimize multi-sample diversity and a single-sample enhancement leveraging semantically critical tokens. By integrating semantic entropy, logit-based scoring, and structural characteristics of function calls, the proposed method substantially improves both the accuracy and practical utility of uncertainty estimation in LLM-driven function invocation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used approach for equipping LLMs with tool-use capabilities. However, an LLM calling functions incorrectly can have severe implications, especially when their effects are irreversible, e.g., transferring money or deleting data. Hence, it is of paramount importance to consider the LLM's confidence that a function call solves the task correctly prior to executing it. Uncertainty Quantification (UQ) methods can be used to quantify this confidence and prevent potentially incorrect function calls. In this work, we present what is, to our knowledge, the first evaluation of UQ methods for LLM Function-Calling (FC). While multi-sample UQ methods, such as Semantic Entropy, show strong performance for natural language Q&A tasks, we find that in the FC setting, it offers no clear advantage over simple single-sample UQ methods. Additionally, we find that the particularities of FC outputs can be leveraged to improve the performance of existing UQ methods in this setting. Specifically, multi-sample UQ methods benefit from clustering FC outputs based on their abstract syntax tree parsing, while single-sample UQ methods can be improved by selecting only semantically meaningful tokens when calculating logit-based uncertainty scores.

Problem

• Uncertainty Quantification
• LLM Function-Calling
• Confidence Estimation
• Tool Use
• Large Language Models

Innovation

• Uncertainty Quantification
• Function-Calling
• Large Language Models
• Abstract Syntax Tree
• Semantic Token Selection