Detecting Ambiguities to Guide Query Rewrite for Robust Conversations in Enterprise AI Assistants

2025-02-01

AI Summary

Enterprise AI assistants frequently suffer from semantic ambiguity in multi-turn dialogues due to coreference resolution failures and context dependency, leading to ambiguous queries and inaccurate responses. To address this, we propose an ambiguity-aware query rewriting framework that formally defines and models the novel task of *ambiguity-guided query rewriting*. We construct a fine-grained ambiguity taxonomy grounded in real-world dialogue logs and design a lightweight hybrid classifier—integrating rule-based heuristics and learned features—for high-precision ambiguity detection and controllable generative rewriting, minimizing unnecessary intervention on unambiguous queries. Our method adopts a joint NLU-NLG architecture to balance efficiency and interpretability. Evaluated on the Adobe Experience Platform AI Assistant, it significantly improves question-answering robustness, effectively mitigates ambiguity-induced errors, and preserves both intent fidelity and response latency for clear queries.

Technology Category

• Natural Language Processing: Question Answering
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-turn conversations with an Enterprise AI Assistant can be challenging due to conversational dependencies in questions, leading to ambiguities and errors. To address this, we propose an NLU-NLG framework for ambiguity detection and resolution through reformulating query automatically and introduce a new task called"Ambiguity-guided Query Rewrite."To detect ambiguities, we develop a taxonomy based on real user conversational logs and draw insights from it to design rules and extract features for a classifier which yields superior performance in detecting ambiguous queries, outperforming LLM-based baselines. Furthermore, coupling the query rewrite module with our ambiguity detecting classifier shows that this end-to-end framework can effectively mitigate ambiguities without risking unnecessary insertions of unwanted phrases for clear queries, leading to an improvement in the overall performance of the AI Assistant. Due to its significance, this has been deployed in the real world application, namely Adobe Experience Platform AI Assistant.

Problem

• AI assistants
• multi-turn dialogue
• ambiguity and errors

Innovation

• NLU-NLG Framework
• Ambiguity Detection
• Query Rewrite