RAC: Retrieval-Augmented Clarification for Faithful Conversational Search

2026-01-16

AI Summary

This work addresses the challenge in conversational search where clarification questions often lack grounding in available evidence, rendering them unanswerable. To tackle this, the authors propose a retrieval-augmented approach that, for the first time, integrates retrieval augmentation with contrastive preference optimization to ensure generated clarification questions are firmly anchored in retrieved document evidence, thereby enhancing both answerability and faithfulness. The method employs multi-strategy index retrieval, fine-tunes large language models to effectively incorporate contextual information, and introduces novel evaluation metrics based on natural language inference (NLI) and data-to-text generation. Extensive experiments across four benchmarks demonstrate significant improvements over existing baselines, with the proposed metrics confirming a consistent increase in the faithfulness of clarification questions to the supporting context.

Technology Category

• Natural Language Processing: Question Answering
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Community question answering

Abstract

Clarification questions help conversational search systems resolve ambiguous or underspecified user queries. While prior work has focused on fluency and alignment with user intent, especially through facet extraction, much less attention has been paid to grounding clarifications in the underlying corpus. Without such grounding, systems risk asking questions that cannot be answered from the available documents. We introduce RAC (Retrieval-Augmented Clarification), a framework for generating corpus-faithful clarification questions. After comparing several indexing strategies for retrieval, we fine-tune a large language model to make optimal use of research context and to encourage the generation of evidence-based question. We then apply contrastive preference optimization to favor questions supported by retrieved passages over ungrounded alternatives. Evaluated on four benchmarks, RAC demonstrate significant improvements over baselines. In addition to LLM-as-Judge assessments, we introduce novel metrics derived from NLI and data-to-text to assess how well questions are anchored in the context, and we demonstrate that our approach consistently enhances faithfulness.

Problem

• conversational search
• clarification questions
• corpus grounding
• faithfulness
• retrieval-augmented

Innovation

• Retrieval-Augmented Clarification
• corpus-faithful
• contrastive preference optimization
• clarification question generation
• conversational search