AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification

2025-03-03

AI Summary

Existing interactive clarification methods for ambiguous user queries in LLM-based tool invocation rely on manually curated data and lack multi-turn error correction mechanisms. Method: We propose the first parameter-driven automated clarification data construction paradigm: (1) generating high-quality multi-turn clarification samples via structured query-tool mapping; (2) introducing selective masking fine-tuning for dynamic error detection and self-correction; and (3) integrating parameter disentanglement reconstruction, intent recovery modeling, and tool trajectory-augmented training. Contribution/Results: Our self-correcting clarification framework requires no human annotation and supports zero-shot API generalization. Experiments show 79.2% accuracy in implicit intent recovery, a 48.3% improvement in clarification efficiency, and zero-shot adaptation performance on unseen APIs comparable to GPT-4—while significantly reducing inference overhead.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in tool learning. In real-world scenarios, user queries are often ambiguous and incomplete, requiring effective clarification. However, existing interactive clarification approaches face two critical limitations: reliance on manually constructed datasets and lack of error correction mechanisms during multi-turn clarification. We present AskToAct, which addresses these challenges by exploiting the structural mapping between queries and their tool invocation solutions. Our key insight is that tool parameters naturally represent explicit user intents. By systematically removing key parameters from queries while retaining them as ground truth, we enable automated construction of high-quality training data. We further enhance model robustness by fine-tuning on error-correction augmented data using selective masking mechanism, enabling dynamic error detection during clarification interactions. Comprehensive experiments demonstrate that AskToAct significantly outperforms existing approaches, achieving above 79% accuracy in recovering critical unspecified intents and enhancing clarification efficiency by an average of 48.34% while maintaining high accuracy in tool invocation. Our framework exhibits robust performance across varying complexity levels and successfully generalizes to entirely unseen APIs without additional training, achieving performance comparable to GPT-4 with substantially fewer computational resources.

Problem

• Addresses ambiguity in user queries for tool use.
• Automates training data construction without manual datasets.
• Enhances error correction in multi-turn clarification interactions.

Innovation

• Automated training data construction via parameter removal
• Error-correction augmented data with selective masking
• Dynamic error detection during multi-turn clarification