Zero-Shot and Efficient Clarification Need Prediction in Conversational Search

2025-02-28

AI Summary

In conversational search, clarifying user intent faces dual challenges: scarcity of labeled data and high inference latency. To address these, we propose Zef-CNP, a zero-shot, efficient framework. First, we introduce TIQ-CoT, a chain-of-thought prompting strategy that integrates topic- and information-need-aware counterfactual query generation (CoQu) to automatically construct high-quality ambiguous–explicit query pairs. Second, we leverage zero-shot large language models (LLMs) to generate chain-of-thought (CoT) supervision signals, guiding the training of a lightweight discriminative model. Zef-CNP requires no human annotation and avoids online LLM inference. Evaluated on multiple benchmarks, it significantly outperforms zero-shot and few-shot LLM baselines—achieving up to 12.6% higher accuracy and reducing inference latency by an order of magnitude. To our knowledge, Zef-CNP is the first method to achieve joint optimization of high accuracy and low latency in intent clarification for conversational search.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Clarification need prediction (CNP) is a key task in conversational search, aiming to predict whether to ask a clarifying question or give an answer to the current user query. However, current research on CNP suffers from the issues of limited CNP training data and low efficiency. In this paper, we propose a zero-shot and efficient CNP framework (Zef-CNP), in which we first prompt large language models (LLMs) in a zero-shot manner to generate two sets of synthetic queries: ambiguous and specific (unambiguous) queries. We then use the generated queries to train efficient CNP models. Zef-CNP eliminates the need for human-annotated clarification-need labels during training and avoids the use of LLMs with high query latency at query time. To further improve the generation quality of synthetic queries, we devise a topic-, information-need-, and query-aware chain-of-thought (CoT) prompting strategy (TIQ-CoT). Moreover, we enhance TIQ-CoT with counterfactual query generation (CoQu), which guides LLMs first to generate a specific/ambiguous query and then sequentially generate its corresponding ambiguous/specific query. Experimental results show that Zef-CNP achieves superior CNP effectiveness and efficiency compared with zero- and few-shot LLM-based CNP predictors.

Problem

• Predicts need for clarification in conversational search efficiently.
• Generates synthetic queries without human-annotated labels.
• Improves query generation quality using advanced prompting strategies.

Innovation

• Zero-shot LLM prompting for synthetic queries
• TIQ-CoT strategy for query generation quality
• Counterfactual query generation (CoQu) enhancement