The PROPER Approach to Proactivity: Benchmarking and Advancing Knowledge Gap Navigation

2026-01-14

AI Summary

This work addresses the limitation of traditional language assistants, which, due to their passive response paradigm, often fail to identify users’ implicit yet relevant needs, leaving critical knowledge gaps unaddressed. To overcome this, the authors propose ProPer, a novel dual-agent proactive interaction framework. It features a Dimension Generation Agent (DGA) that extracts implicit contextual dimensions from user data and a Response Generation Agent (RGA) that synthesizes both explicit and implicit information to deliver timely, personalized interventions. The framework incorporates implicit dimension generation and a multi-dimensional reranking mechanism, alongside a structured evaluation protocol tailored to knowledge gap resolution. Experimental results demonstrate that ProPer significantly enhances response quality across multiple domains, achieving up to an 84% improvement in single-turn evaluations and consistently outperforming baseline methods in multi-turn interactions.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Humans and AI: Intelligent User Interfaces

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Most language-based assistants follow a reactive ask-and-respond paradigm, requiring users to explicitly state their needs. As a result, relevant but unexpressed needs often go unmet. Existing proactive agents attempt to address this gap either by eliciting further clarification, preserving this burden, or by extrapolating future needs from context, often leading to unnecessary or mistimed interventions. We introduce ProPer, Proactivity-driven Personalized agents, a novel two-agent architecture consisting of a Dimension Generating Agent (DGA) and a Response Generating Agent (RGA). DGA, a fine-tuned LLM agent, leverages explicit user data to generate multiple implicit dimensions (latent aspects relevant to the user's task but not considered by the user) or knowledge gaps. These dimensions are selectively filtered using a reranker based on quality, diversity, and task relevance. RGA then balances explicit and implicit dimensions to tailor personalized responses with timely and proactive interventions. We evaluate ProPer across multiple domains using a structured, gap-aware rubric that measures coverage, initiative appropriateness, and intent alignment. Our results show that ProPer improves quality scores and win rates across all domains, achieving up to 84% gains in single-turn evaluation and consistent dominance in multi-turn interactions.

Problem

• proactivity
• knowledge gap
• personalized agents
• implicit needs
• language-based assistants

Innovation

• proactive AI
• knowledge gap navigation
• two-agent architecture
• implicit dimension generation
• personalized response generation