ECLAIR: Enhanced Clarification for Interactive Responses in an Enterprise AI Assistant

2025-03-19

AI Summary

Ambiguity in user queries poses a significant challenge for enterprise-grade AI assistants, hindering accurate response generation. Method: This paper proposes a multi-agent interactive clarification framework that leverages domain knowledge–enhanced, specialized agents to collaboratively identify ambiguity, generate context-aware clarification questions, and iteratively refine responses based on real-time user feedback. Unlike static prompting or single-agent approaches, it introduces a dynamic, progressive ambiguity resolution paradigm tailored to enterprise settings, integrating prompt engineering, interaction-driven optimization, and domain-adaptive clarification generation. Contribution/Results: Evaluated on real-world customer data, the framework significantly improves the relevance and practicality of clarification questions over few-shot baselines, leading to a substantial increase in final response accuracy. This work establishes a novel, scalable approach to query disambiguation in complex, domain-specific enterprise environments.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Intelligent Query Processing
• Humans and AI: Intelligent User Interfaces

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large language models (LLMs) have shown remarkable progress in understanding and generating natural language across various applications. However, they often struggle with resolving ambiguities in real-world, enterprise-level interactions, where context and domain-specific knowledge play a crucial role. In this demonstration, we introduce ECLAIR (Enhanced CLArification for Interactive Responses), a multi-agent framework for interactive disambiguation. ECLAIR enhances ambiguous user query clarification through an interactive process where custom agents are defined, ambiguity reasoning is conducted by the agents, clarification questions are generated, and user feedback is leveraged to refine the final response. When tested on real-world customer data, ECLAIR demonstrates significant improvements in clarification question generation compared to standard few-shot methods.

Problem

• Resolving ambiguities in enterprise AI interactions
• Enhancing query clarification through multi-agent framework
• Improving clarification questions with user feedback

Innovation

• Multi-agent framework for interactive disambiguation
• Custom agents conduct ambiguity reasoning
• Leverages user feedback to refine responses