Well Begun is Half Done: Training-Free and Model-Agnostic Semantically Guaranteed User Representation Initialization for Multimodal Recommendation

2026-04-16

AI Summary

This work addresses the semantic gap in multimodal recommendation systems, where user representations are typically initialized without semantic information, leading to a significant mismatch with modality-rich item representations. To bridge this gap, the authors propose SG-URInit, a training-free and model-agnostic user representation initialization method that fuses local modality features from a user’s interacted items with global semantic features derived from the clusters to which those items belong. By integrating both local and global semantics, SG-URInit provides semantically enriched initial user embeddings. This approach introduces, for the first time, a semantics-aware initialization mechanism in multimodal recommendation, consistently enhancing the performance of state-of-the-art models across multiple real-world datasets, accelerating convergence, and effectively alleviating the item cold-start problem.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Recent advancements in multimodal recommendations, which leverage diverse modality information to mitigate data sparsity and improve recommendation accuracy, have gained significant attention. However, existing multimodal recommendations overlook the critical role of user representation initialization. Unlike items, which are naturally associated with rich modality information, users lack such inherent information. Consequently, item representations initialized based on meaningful modality information and user representations initialized randomly exhibit a significant semantic gap. To this end, we propose a Semantically Guaranteed User Representation Initialization (SG-URInit). SG-URInit constructs the initial representation for each user by integrating both the modality features of the items they have interacted with and the global features of their corresponding clusters. SG-URInit enables the initialization of semantically enriched user representations that effectively capture both local (item-level) and global (cluster-level) semantics. Our SG-URInit is training-free and model-agnostic, meaning it can be seamlessly integrated into existing multimodal recommendation models without incurring any additional computational overhead during training. Extensive experiments on multiple real-world datasets demonstrate that incorporating SG-URInit into advanced multimodal recommendation models significantly enhances recommendation performance. Furthermore, the results show that SG-URInit can further alleviate the item cold-start problem and also accelerate model convergence, making it an efficient and practical solution for multimodal recommendations.

Problem

• multimodal recommendation
• user representation initialization
• semantic gap
• data sparsity
• cold-start problem

Innovation

• user representation initialization
• multimodal recommendation
• training-free
• model-agnostic
• semantic alignment