Prompt Tuning as User Inherent Profile Inference Machine

2024-08-13

arXiv.org

AI Summary

To address text noise in large language models (LLMs) for recommendation—stemming from unstable instruction following, modality gaps between textual and collaborative signals, and high inference latency—this paper proposes UserIP-Tuning. It reformulates prompt tuning as a causal inference task over implicit user profiles, leveraging an Expectation-Maximization (EM) algorithm to disentangle users’ intrinsic profiles from behavioral sequences. Crucially, it introduces a profile quantization codebook that maps continuous profile embeddings into discrete collaborative IDs, thereby bridging the modality gap and enabling lightweight online deployment. Evaluated on four public benchmarks, UserIP-Tuning achieves significant improvements over state-of-the-art methods: 52% reduction in inference latency, 37% lower memory footprint, enhanced robustness, cross-domain transferability, and millisecond-scale response time.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large Language Models (LLMs) have exhibited significant promise in recommender systems by empowering user profiles with their extensive world knowledge and superior reasoning capabilities. However, LLMs face challenges like unstable instruction compliance, modality gaps, and high inference latency, leading to textual noise and limiting their effectiveness in recommender systems. To address these challenges, we propose UserIP-Tuning, which uses prompt-tuning to infer user profiles. It integrates the causal relationship between user profiles and behavior sequences into LLMs' prompts. And employs expectation maximization to infer the embedded latent profile, minimizing textual noise by fixing the prompt template. Furthermore, A profile quantization codebook bridges the modality gap by categorizing profile embeddings into collaborative IDs, which are pre-stored for online deployment. This improves time efficiency and reduces memory usage. Experiments on four public datasets show that UserIP-Tuning outperforms state-of-the-art recommendation algorithms. Additional tests and case studies confirm its effectiveness, robustness, and transferability.

Problem

• Inferring user profiles from behavior sequences using prompt-tuning
• Reducing textual noise and modality gaps in LLM-based recommendations
• Improving inference efficiency for real-world recommendation systems

Innovation

• Uses prompt-tuning to infer user profiles
• Employs Expectation Maximization to minimize textual noise
• Applies profile quantization codebook to bridge modality gaps