LLM Agents Already Know When to Call Tools -- Even Without Reasoning

2026-05-09

AI Summary

This work addresses the inefficiency of large language model (LLM) agents that frequently invoke external tools even when unnecessary, leading to significant resource waste, and highlights the absence of a systematic benchmark for evaluating tool-call necessity. To this end, the authors introduce When2Tool, a novel benchmark spanning 18 diverse environments, which reveals for the first time that LLMs’ hidden states implicitly encode signals indicating whether a tool is needed—though these signals remain underutilized. Building on this insight, they propose Probe&Prefill, a training-free method that employs a lightweight linear probe to detect tool necessity and guides generation accordingly. Experiments demonstrate that Probe&Prefill reduces tool calls by 48% on average while incurring only a 1.7% drop in task accuracy, substantially outperforming existing strategies, which either reduce calls by merely 6% at comparable accuracy or suffer over five times greater accuracy loss for similar call reduction.

Abstract

Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark systematically studies when a tool call is actually needed. We propose When2Tool, a benchmark of 18 environments (15 single-hop, 3 multi-hop) spanning three categories of tool necessity -- computational scale, knowledge boundaries, and execution reliability -- each with controlled difficulty levels that create a clear decision boundary between tool-necessary and tool-unnecessary tasks. We evaluate two families of training-free baselines: Prompt-only (varying the prompt to discourage unnecessary calls) and Reason-then-Act (requiring the model to reason about tool necessity before acting). Both provide limited control: Prompt-only suppresses necessary calls alongside unnecessary ones, and Reason-then-Act still incurs a disproportionate accuracy cost on hard tasks. To understand why these baselines fail, we probe the models' hidden states and find that tool necessity is linearly decodable from the pre-generation representation with AUROC 0.89--0.96 across six models, substantially exceeding the model's own verbalized reasoning. This reveals that models already know when tools are needed, but fail to act on this knowledge during generation. Building on this finding, we propose Probe&Prefill, which uses a lightweight linear probe to read the hidden-state signal and prefills the model's response with a steering sentence. Across all models tested, Probe&Prefill reduces tool calls by 48% with only 1.7% accuracy loss, while the best baseline at comparable accuracy only reduces 6% of tool calls, or achieves a similar tool call reduction but incurs a 5$\times$ higher accuracy loss. Our code is available at https://github.com/Trustworthy-ML-Lab/when2tool

Problem

• tool calling
• large language models
• tool necessity
• benchmark
• efficiency

Innovation

• tool necessity detection
• hidden state probing
• Probe&Prefill
• LLM tool calling
• training-free control