Proactive User Information Acquisition via Chats on User-Favored Topics

2025-04-10

AI Summary

This paper addresses the challenge of naturally and unobtrusively eliciting user responses to critical questions within their preferred topics in beneficial dialogue systems. To this end, we formally introduce the PIVOT task—the first dedicated formulation of proactive information acquisition under topic alignment. We empirically demonstrate that mainstream large language models perform poorly on this task. To bridge this gap, we construct the first high-quality, human-annotated dataset specifically designed for PIVOT. Leveraging data-driven insights, we propose a lightweight yet effective approach integrating dialogue policy modeling, topic consistency control, and progressive information prompting. Extensive experiments on our benchmark show substantial improvements in information acquisition success rate, validating the efficacy of topic-integrated elicitation strategies. Our work establishes a novel paradigm and foundational methodology for deploying practical, user-centered proactive dialogue systems.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Information Extraction
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Chat-oriented dialogue systems designed to provide tangible benefits, such as sharing the latest news or preventing frailty in senior citizens, often require Proactive acquisition of specific user Information via chats on user-faVOred Topics (PIVOT). This study proposes the PIVOT task, designed to advance the technical foundation for these systems. In this task, a system needs to acquire the answers of a user to predefined questions without making the user feel abrupt while engaging in a chat on a predefined topic. We found that even recent large language models (LLMs) show a low success rate in the PIVOT task. We constructed a dataset suitable for the analysis to develop more effective systems. Finally, we developed a simple but effective system for this task by incorporating insights obtained through the analysis of this dataset.

Problem

• Develop chat systems for proactive user information acquisition
• Enhance dialogue systems to avoid abrupt user interactions
• Improve LLM success rates in targeted information gathering

Innovation

• Proactive user info acquisition via favored topics
• Dataset construction for PIVOT task analysis
• Simple effective system using dataset insights