How You Ask Matters! Adaptive RAG Robustness to Query Variations

2026-04-12

AI Summary

This study addresses the fragility of Adaptive Retrieval-Augmented Generation (Adaptive RAG) systems when confronted with semantically equivalent queries that differ in surface form, which leads to significant fluctuations in answer quality, retrieval decisions, and computational overhead. To systematically evaluate the robustness of Adaptive RAG across multiple dimensions, the authors construct the first large-scale benchmark of semantically equivalent query variants, combining human rewrites with large language model–generated paraphrases. Their analysis reveals that even minor syntactic or lexical variations can substantially alter system behavior, and notably, larger language models do not consistently yield improved robustness—highlighting a critical limitation in current Adaptive RAG methodologies.

Technology Category

• Natural Language Processing: Safety and Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Adaptive Retrieval-Augmented Generation (RAG) promises accuracy and efficiency by dynamically triggering retrieval only when needed and is widely used in practice. However, real-world queries vary in surface form even with the same intent, and their impact on Adaptive RAG remains under-explored. We introduce the first large-scale benchmark of diverse yet semantically identical query variations, combining human-written and model-generated rewrites. Our benchmark facilitates a systematic evaluation of Adaptive RAG robustness by examining its key components across three dimensions: answer quality, computational cost, and retrieval decisions. We discover a critical robustness gap, where small surface-level changes in queries dramatically alter retrieval behavior and accuracy. Although larger models show better performance, robustness does not improve accordingly. These findings reveal that Adaptive RAG methods are highly vulnerable to query variations that preserve identical semantics, exposing a critical robustness challenge.

Problem

• Adaptive RAG
• query variations
• robustness
• retrieval-augmented generation
• semantic equivalence

Innovation

• Adaptive RAG
• query variations
• robustness
• retrieval-augmented generation
• benchmark