Response-Aware User Memory Selection for LLM Personalization

2026-04-15

AI Summary

Existing personalization approaches for large language models predominantly rely on semantic similarity to select user memories, overlooking the actual impact of these memories on the model’s output distribution. This work proposes RUMS, a novel method that introduces mutual information from information theory into memory selection for the first time. By quantifying the mutual information between subsets of user memories and model responses, RUMS identifies memory items that maximally reduce uncertainty in the output distribution. The approach significantly enhances response quality while reducing computational overhead by up to 95%. It outperforms current state-of-the-art methods and even surpasses models with 400 times more parameters, achieving more efficient and human-preference-aligned personalized generation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

A common approach to personalization in large language models (LLMs) is to incorporate a subset of the user memory into the prompt at inference time to guide the model's generation. Existing methods select these subsets primarily using similarity between user memory items and input queries, ignoring how features actually affect the model's response distribution. We propose Response-Utility optimization for Memory Selection (RUMS), a novel method that selects user memory items by measuring the mutual information between a subset of memory and the model's outputs, identifying items that reduce response uncertainty and sharpen predictions beyond semantic similarity. We demonstrate that this information-theoretic foundation enables more principled user memory selection that aligns more closely with human selection compared to state-of-the-art methods, and models $400\times$ larger. Additionally, we show that memory items selected using RUMS result in better response quality compared to existing approaches, while having up to $95\%$ reduction in computational cost.

Problem

• LLM Personalization
• User Memory Selection
• Response Distribution
• Semantic Similarity
• Mutual Information

Innovation

• memory selection
• mutual information
• LLM personalization
• response uncertainty
• information-theoretic optimization