What Makes a Good Query? Measuring the Impact of Human-Confusing Linguistic Features on LLM Performance

2026-02-23

AI Summary

This study investigates how the linguistic characteristics of user queries systematically influence the hallucination propensity of large language models (LLMs). Drawing on established linguistic theory, the authors construct—for the first time—a quantifiable and interpretable 22-dimensional feature vector to characterize queries. Through large-scale empirical analysis of 369,837 real-world queries, complemented by cross-model and cross-dataset comparisons, the work uncovers the mechanistic relationship between query structure and hallucination risk. The findings reveal that deep clause nesting and ambiguous referential expressions significantly increase the likelihood of hallucinations, whereas queries with clear intent and high answerability effectively mitigate such errors. These insights provide a theoretical foundation and practical guidance for designing query-rewriting strategies and low-hallucination interactive systems.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large Language Model (LLM) hallucinations are usually treated as defects of the model or its decoding strategy. Drawing on classical linguistics, we argue that a query's form can also shape a listener's (and model's) response. We operationalize this insight by constructing a 22-dimension query feature vector covering clause complexity, lexical rarity, and anaphora, negation, answerability, and intention grounding, all known to affect human comprehension. Using 369,837 real-world queries, we ask: Are there certain types of queries that make hallucination more likely? A large-scale analysis reveals a consistent"risk landscape": certain features such as deep clause nesting and underspecification align with higher hallucination propensity. In contrast, clear intention grounding and answerability align with lower hallucination rates. Others, including domain specificity, show mixed, dataset- and model-dependent effects. Thus, these findings establish an empirically observable query-feature representation correlated with hallucination risk, paving the way for guided query rewriting and future intervention studies.

Problem

• query features
• LLM hallucination
• linguistic complexity
• answerability
• intention grounding

Innovation

• query features
• hallucination risk
• linguistic complexity
• intention grounding
• large language models