Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalization

2026-01-17

AI Summary

Existing user history retrieval methods based on semantic relevance often fail to effectively enhance the personalized generation quality of large language models and can even degrade performance by introducing redundant or conflicting information. This work proposes PURPLE, a novel framework that formulates user profiling as a set generation problem. By integrating contextual bandits with the Plackett–Luce ranking model, PURPLE explicitly captures dependencies among user records and leverages the likelihood of reference responses as dense feedback to directly align retrieval and generation objectives. Moving beyond conventional paradigms that rely solely on semantic similarity, PURPLE achieves state-of-the-art performance across nine personalized tasks, demonstrating superior effectiveness and efficiency, and validating its strong generalizability and scalability.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) excel at general-purpose tasks, yet adapting their responses to individual users remains challenging. Retrieval augmentation provides a lightweight alternative to fine-tuning by conditioning LLMs on user history records, and existing approaches typically select these records based on semantic relevance. We argue that relevance serves as an unreliable proxy for utility: a record may be semantically similar to a query yet fail to improve generation quality or even degrade it due to redundancy or conflicting information. To bridge this gap, we propose PURPLE, a contextual bandit framework that oPtimizes UseR Profiles for Llm pErsonalization. In contrast to a greedy selection of the most relevant records, PURPLE treats profile construction as a set generation process and utilizes a Plackett-Luce ranking model to capture complex inter-record dependencies. By training with dense feedback provided by the likelihood of the reference response, our method aligns retrieval directly with generation quality. Extensive experiments on nine personalization tasks demonstrate that PURPLE consistently outperforms strong heuristic and retrieval-augmented baselines in both effectiveness and efficiency, establishing a principled and scalable solution for optimizing user profiles.

Problem

• Retrieval-Augmented LLM
• Personalization
• User Profiles
• Contextual Bandits
• Semantic Relevance

Innovation

• Contextual Bandits
• Retrieval-Augmented LLM
• User Profile Optimization
• Plackett-Luce Model
• Personalization