Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations

2024-05-30

arXiv.org

AI Summary

To address the recommendation challenge for cold-start users—such as newly registered users—who lack historical behavioral data, this paper proposes KALM4Rec, a keyword-driven retrieval-augmented large language model framework. The method operates in two stages: first, candidate restaurants are retrieved based on minimal user-provided keywords; second, a large language model (LLM) performs re-ranking using zero-shot or few-shot prompting augmented with explainability instructions. Its core innovation lies in the first integration of explainability instructions into contextual prompts to enhance both recommendation accuracy and trustworthiness. Additionally, the authors introduce a novel multi-city Yelp benchmark specifically designed for cold-start evaluation. Experiments on a three-city Yelp restaurant dataset demonstrate that KALM4Rec achieves an 18.7% improvement in NDCG@10 over baselines, along with significant gains in accuracy and diversity.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent advancements in Large Language Models (LLMs) have shown significant potential in enhancing recommender systems. However, addressing the cold-start recommendation problem, where users lack historical data, remains a considerable challenge. In this paper, we introduce KALM4Rec (Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations), a novel framework specifically designed to tackle this problem by requiring only a few input keywords from users in a practical scenario of cold-start user restaurant recommendations. KALM4Rec operates in two main stages: candidates retrieval and LLM-based candidates re-ranking. In the first stage, keyword-driven retrieval models are used to identify potential candidates, addressing LLMs' limitations in processing extensive tokens and reducing the risk of generating misleading information. In the second stage, we employ LLMs with various prompting strategies, including zero-shot and few-shot techniques, to re-rank these candidates by integrating multiple examples directly into the LLM prompts. Our evaluation, using a Yelp restaurant dataset with user reviews from three English-speaking cities, shows that our proposed framework significantly improves recommendation quality. Specifically, the integration of in-context instructions with LLMs for re-ranking markedly enhances the performance of the cold-start user recommender system.

Problem

• Addresses cold-start user recommendations with minimal input keywords
• Combines keyword-driven retrieval and LLM-based re-ranking for accuracy
• Improves recommendation quality using in-context LLM instructions

Innovation

• Keyword-driven retrieval models for initial candidate selection
• LLM-based re-ranking with zero-shot and few-shot prompting
• Integration of in-context instructions to enhance recommendations