Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations

2026-04-13

AI Summary

This work identifies and formally names a previously unrecognized structural alignment bias in large language models, which leads them to erroneously invoke tools even when those tools are semantically irrelevant to the user query. To investigate this phenomenon, the authors construct SABEval, a novel dataset designed to disentangle structural alignment from semantic relevance. They further propose a contrastive attention attribution method to elucidate the underlying mechanisms driving this bias and introduce a pathway re-balancing training strategy to mitigate it. Experimental results demonstrate that the proposed approach significantly reduces erroneous tool invocations while preserving the model’s general capability for appropriate tool use.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in utilizing external tools. In practice, however, LLMs are often exposed to tools that are irrelevant to the user's query, in which case the desired behavior is to refrain from invocations. In this work, we identify a widespread yet overlooked mechanistic flaw in tool refusal, which we term structural alignment bias: Even when a tool fails to serve the user's goal, LLMs still tend to invoke it whenever query attributes can be validly assigned to tool parameters. To systematically study this bias, we introduce SABEval, a new dataset that decouples structural alignment from semantic relevance. Our analysis shows that structural alignment bias induces severe tool-invocation errors in LLMs, yet remains largely unaccounted for in existing evaluations. To investigate the internal mechanisms underlying this bias, we propose Contrastive Attention Attribution, which reveals two competing pathways for semantic checking and structural matching. The relative strength of these pathways drives LLMs' tool invocation decisions. Based on these findings, we further introduce a rebalancing strategy that effectively mitigates structural alignment bias, as demonstrated by extensive experiments, without degrading general tool-use capabilities.

Problem

• structural alignment bias
• tool irrelevance
• tool invocation
• large language models
• semantic relevance

Innovation

• structural alignment bias
• tool invocation
• Contrastive Attention Attribution
• SABEval
• large language models