CLEAR-KGQA: Clarification-Enhanced Ambiguity Resolution for Knowledge Graph Question Answering

2025-04-13

AI Summary

To address pervasive entity ambiguity (e.g., confounding of entities with similar names) and intent ambiguity (e.g., query polysemy) in Knowledge Graph Question Answering (KGQA), this paper proposes the first end-to-end ambiguity-aware multi-turn interactive KGQA framework. Methodologically, it innovatively integrates Bayesian uncertainty modeling with a dual-agent collaborative mechanism—comprising an LLM-driven user simulator and a QA agent—to dynamically detect ambiguities and adaptively initiate clarification dialogues; it further incorporates iterative logical form refinement to enhance reasoning robustness. Key contributions include: (1) the first ambiguity-resolution query dataset constructed from real-world interaction histories; and (2) significant accuracy improvements on WebQSP and ComplexWebQuestions (CWQ), effectively mitigating both named-entity conflicts and query intent ambiguity.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Intelligent Query Processing
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web question answering

Abstract

This study addresses the challenge of ambiguity in knowledge graph question answering (KGQA). While recent KGQA systems have made significant progress, particularly with the integration of large language models (LLMs), they typically assume user queries are unambiguous, which is an assumption that rarely holds in real-world applications. To address these limitations, we propose a novel framework that dynamically handles both entity ambiguity (e.g., distinguishing between entities with similar names) and intent ambiguity (e.g., clarifying different interpretations of user queries) through interactive clarification. Our approach employs a Bayesian inference mechanism to quantify query ambiguity and guide LLMs in determining when and how to request clarification from users within a multi-turn dialogue framework. We further develop a two-agent interaction framework where an LLM-based user simulator enables iterative refinement of logical forms through simulated user feedback. Experimental results on the WebQSP and CWQ dataset demonstrate that our method significantly improves performance by effectively resolving semantic ambiguities. Additionally, we contribute a refined dataset of disambiguated queries, derived from interaction histories, to facilitate future research in this direction.

Problem

• Resolving entity and intent ambiguity in KGQA
• Dynamic clarification via Bayesian inference and LLMs
• Improving KGQA performance through multi-turn dialogue

Innovation

• Dynamic ambiguity resolution via interactive clarification
• Bayesian inference quantifies and guides query ambiguity
• Two-agent framework refines logical forms iteratively