Heterogeneous User Modeling for LLM-based Recommendation

2025-07-06

AI Summary

To address core challenges in open-domain recommendation—including difficulty modeling cross-domain heterogeneous user behaviors, weak generalization, sensitivity to interaction noise, and inter-domain imbalance (“seesaw effect”)—this paper proposes a robust heterogeneous user modeling framework. Methodologically: (1) it introduces customized prompts and behavior sequence masking to efficiently compress multi-source heterogeneous interactions; (2) it incorporates a learnable domain importance scoring module to dynamically weight and fuse cross-domain representations, explicitly mitigating competitive inter-domain imbalance. Technically, the approach integrates large language models (LLMs), prompt engineering, and domain-aware weighting strategies. Extensive experiments on multi-source heterogeneous datasets demonstrate significant improvements in recommendation accuracy and cross-domain generalization, alongside enhanced robustness to sparse and noisy interactions. This work establishes a novel paradigm for LLM-driven open-domain user modeling.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Humanities & Computational Social Science
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Leveraging Large Language Models (LLMs) for recommendation has demonstrated notable success in various domains, showcasing their potential for open-domain recommendation. A key challenge to advancing open-domain recommendation lies in effectively modeling user preferences from users' heterogeneous behaviors across multiple domains. Existing approaches, including ID-based and semantic-based modeling, struggle with poor generalization, an inability to compress noisy interactions effectively, and the domain seesaw phenomenon. To address these challenges, we propose a Heterogeneous User Modeling (HUM) method, which incorporates a compression enhancer and a robustness enhancer for LLM-based recommendation. The compression enhancer uses a customized prompt to compress heterogeneous behaviors into a tailored token, while a masking mechanism enhances cross-domain knowledge extraction and understanding. The robustness enhancer introduces a domain importance score to mitigate the domain seesaw phenomenon by guiding domain optimization. Extensive experiments on heterogeneous datasets validate that HUM effectively models user heterogeneity by achieving both high efficacy and robustness, leading to superior performance in open-domain recommendation.

Problem

• Modeling user preferences from heterogeneous behaviors across domains
• Overcoming poor generalization and noisy interaction compression
• Mitigating domain seesaw phenomenon in open-domain recommendation

Innovation

• Customized prompt compresses heterogeneous behaviors
• Masking mechanism enhances cross-domain knowledge
• Domain importance score mitigates seesaw phenomenon