Collaborative Retrieval for Large Language Model-based Conversational Recommender Systems

2025-02-19

AI Summary

To address the challenge of effectively integrating user behavioral data into large language models (LLMs) for conversational recommendation systems (CRS), this paper proposes CRAG—a Collaborative Retrieval-Augmented Generation framework that enables end-to-end joint modeling of LLMs and collaborative filtering (CF) for the first time. CRAG introduces a collaborative retrieval mechanism that dynamically injects CF-derived user-item interaction signals into both the retrieval and generation stages of the LLM, while jointly leveraging dialogue state and multi-source data to enhance personalized representation learning. Experiments on Reddit-v2 and ReDial demonstrate that CRAG significantly improves recommendation accuracy (e.g., +12.3% Recall@5) and item coverage (+18.7%), with particularly strong performance on long-tail items such as newly released movies. CRAG establishes a novel, interpretable, and scalable behavior-aware paradigm for LLM-driven CRS.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Conversational recommender systems (CRS) aim to provide personalized recommendations via interactive dialogues with users. While large language models (LLMs) enhance CRS with their superior understanding of context-aware user preferences, they typically struggle to leverage behavioral data, which have proven to be important for classical collaborative filtering (CF)-based approaches. For this reason, we propose CRAG, Collaborative Retrieval Augmented Generation for LLM-based CRS. To the best of our knowledge, CRAG is the first approach that combines state-of-the-art LLMs with CF for conversational recommendations. Our experiments on two publicly available movie conversational recommendation datasets, i.e., a refined Reddit dataset (which we name Reddit-v2) as well as the Redial dataset, demonstrate the superior item coverage and recommendation performance of CRAG, compared to several CRS baselines. Moreover, we observe that the improvements are mainly due to better recommendation accuracy on recently released movies. The code and data are available at https://github.com/yaochenzhu/CRAG.

Problem

• Enhance LLM-based CRS with collaborative filtering
• Improve recommendation accuracy for new items
• Integrate behavioral data into LLM recommendations

Innovation

• LLMs combined with CF
• CRAG enhances recommendation accuracy
• Improved coverage for new items