Ask to Be Sure: Informative Interactions for Confident Multi-Turn LLM Recommendation

2026-08-16

AI Summary

This study addresses the inefficiency of interaction guidance and the lack of quantifiable information gain in multi-turn conversational recommendation. We propose fine-tuning large language models using recommendation entropy reduction as an unsupervised reward signal, thereby formulating uncertainty minimization as the optimization objective. By integrating Supervised Fine-Tuning (SFT) with Direct Preference Optimization (DPO), this approach facilitates strategic dialogue generation without requiring ground-truth labels. Experiments on the INSPIRED and ReDial datasets demonstrate that the proposed framework significantly enhances both recommendation quality and dialogue confidence. Consequently, this method effectively resolves exploration challenges in multi-turn interactions, establishing a novel paradigm for proactive engagement within conversational recommender systems.

Abstract

Recent advances in large language models (LLMs) have enabled their use as conversational recommender systems (CRS), demonstrating strong recommendation accuracy and natural dialogue. However, guiding multi-turn interactions to elicit user preferences effectively remains challenging. Existing approaches either use separate reinforcement learning agents with templated interactions or optimize for interactivity judged by another LLM, without measuring how much useful information is actually gained. We propose a new approach that quantifies the effectiveness of each interaction by the reduction in the assistant's uncertainty, measured via entropy over recommendations. We apply this entropy reduction as a reward---without relying on ground-truth recommendations, which are often unavailable in real-world scenarios---to fine-tune the LLM, enabling strategic interaction generation. Empirical results with supervised fine-tuning (SFT) and direct preference optimization (DPO) on the INSPIRED and ReDial datasets show that our method improves both recommendation quality and conversational efficiency.

Problem

• Conversational Recommender Systems
• Multi-turn Interaction
• User Preference Elicitation
• Information Gain
• Large Language Models

Innovation

• Entropy Reduction
• Uncertainty Quantification
• Reward Signal
• Multi-Turn Conversational Recommendation
• Direct Preference Optimization