Leveraging Memory Retrieval to Enhance LLM-based Generative Recommendation

2024-12-23

arXiv.org

AI Summary

To address the limitations of large language models (LLMs) in generative recommendation—namely, constrained context windows and difficulty in modeling users’ long-term interests—this paper proposes AutoMR, an automatic memory retrieval framework. AutoMR integrates memory networks with retrieval-augmented generation (RAG) to enable structured storage, semantic alignment, and on-demand retrieval of users’ long-term interaction histories; it further incorporates dynamically retrieved interest representations into LLM-based generation via prompt engineering. To our knowledge, this is the first work to systematically introduce an updatable, retrievable long-term memory mechanism into the LLM-based recommendation paradigm. Extensive experiments on two real-world datasets demonstrate that AutoMR significantly improves Recall@10 and NDCG@10, empirically validating the substantial performance gains achieved by explicit long-term interest modeling in generative recommendation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Leveraging Large Language Models (LLMs) to harness user-item interaction histories for item generation has emerged as a promising paradigm in generative recommendation. However, the limited context window of LLMs often restricts them to focusing on recent user interactions only, leading to the neglect of long-term interests involved in the longer histories. To address this challenge, we propose a novel Automatic Memory-Retrieval framework (AutoMR), which is capable of storing long-term interests in the memory and extracting relevant information from it for next-item generation within LLMs. Extensive experimental results on two real-world datasets demonstrate the effectiveness of our proposed AutoMR framework in utilizing long-term interests for generative recommendation.

Problem

• Enhancing LLM-based recommendation with memory retrieval
• Addressing limited context window in generative recommendation
• Utilizing long-term user interests for item generation

Innovation

• Uses memory retrieval for long-term interest storage
• Extracts relevant info from memory for item generation
• Enhances LLM-based recommendation with AutoMR framework