Disambiguation in Conversational Question Answering in the Era of LLM: A Survey

2025-05-18

AI Summary

Ambiguity in natural language remains a core challenge to the robustness and reliability of large language model (LLM)-driven conversational question answering (CQA). This paper introduces the first comprehensive ambiguity taxonomy tailored to the LLM era, systematically characterizing its origins, categories, and downstream impacts. Methodologically, it unifies and critically compares diverse disambiguation strategies—including context-aware modeling, prompt engineering, retrieval-augmented disambiguation, and multi-turn state tracking—highlighting their respective limitations. Empirical evaluation is conducted on established public benchmarks, and an extensible research roadmap for ambiguity resolution is proposed. The contributions encompass: (1) a formal definitional framework for CQA ambiguity; (2) a systematic methodological survey; (3) a curated overview of relevant datasets; (4) principled evaluation criteria; and (5) a delineation of open challenges. Collectively, this work provides both theoretical foundations and practical guidance for developing more robust, interpretable, and reliable conversational AI systems.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Ambiguity remains a fundamental challenge in Natural Language Processing (NLP) due to the inherent complexity and flexibility of human language. With the advent of Large Language Models (LLMs), addressing ambiguity has become even more critical due to their expanded capabilities and applications. In the context of Conversational Question Answering (CQA), this paper explores the definition, forms, and implications of ambiguity for language driven systems, particularly in the context of LLMs. We define key terms and concepts, categorize various disambiguation approaches enabled by LLMs, and provide a comparative analysis of their advantages and disadvantages. We also explore publicly available datasets for benchmarking ambiguity detection and resolution techniques and highlight their relevance for ongoing research. Finally, we identify open problems and future research directions, proposing areas for further investigation. By offering a comprehensive review of current research on ambiguities and disambiguation with LLMs, we aim to contribute to the development of more robust and reliable language systems.

Problem

• Addressing ambiguity challenges in Conversational Question Answering using LLMs
• Categorizing and comparing LLM-based disambiguation approaches in NLP
• Identifying open problems and future research directions for ambiguity resolution

Innovation

• Categorizes disambiguation approaches using LLMs
• Analyzes datasets for ambiguity detection benchmarking
• Identifies open problems in LLM-based disambiguation