Personalization and Evaluation of Conversational Information Access

2026-06-11

AI Summary

This work addresses three core challenges in conversational information seeking—modeling personal context, generating personalized responses, and automatic evaluation—by proposing an integrated solution. It introduces the ConEL dataset and the CREL method for conversational entity linking, constructs LAPS, a large-scale personalized dialogue dataset, and presents FACE, the first reference-free, globally aware automatic evaluation framework. The proposed approaches substantially enhance the system’s ability to understand and leverage users’ personal contexts, yielding responses better aligned with individual preferences. Moreover, FACE demonstrates high agreement with human judgments, establishing a new paradigm for both the development and evaluation of personalized dialogue systems.

Abstract

Conversational interactions have reshaped information retrieval systems, as users increasingly favour direct answers over traditional hyperlinks. To build reliable Conversational Information Access (CIA) systems that account for personal context, this thesis addresses challenges: (1) personal context extraction, (2) personalized response generation, and (3) effective and interpretable system evaluation. First, we tackle personal context extraction by studying what Entity Linking (EL) in conversations entails, introducing a dataset for conversational entity linking (ConEL), and proposing CREL, a novel EL method tailored for conversational settings. Second, we focus on personalized response generation by proposing LAPS, a method for efficiently constructing large-scale, human-written, personalized conversational datasets, and using them to study how users' preferences can be utilized to generate personalized responses. Finally, we address the need for effective and interpretable system evaluation by introducing FACE, an automatic, reference-free method that assesses entire conversations and aligns closely with human judgments.

Problem

• Conversational Information Access
• Personalization
• Entity Linking
• Response Generation
• System Evaluation

Innovation

• Conversational Entity Linking
• Personalized Response Generation
• Reference-Free Evaluation
• ConEL Dataset
• LAPS Framework