DiffCL: A Diffusion-Based Contrastive Learning Framework with Semantic Alignment for Multimodal Recommendations

2025-01-02

AI Summary

To address three key challenges in multimodal recommendation—data sparsity, modality noise, and cross-modal semantic inconsistency—this paper proposes the first multimodal recommendation framework integrating diffusion models with contrastive learning. Specifically, it employs a diffusion model to generate robust contrastive views that suppress modality-specific noise; introduces an ID-embedding-driven image-text semantic alignment mechanism to ensure cross-modal representation consistency; and constructs an item-relation graph to enrich feature representation and alleviate data sparsity. Extensive experiments on three public benchmarks demonstrate that the proposed method achieves average improvements of 4.2% in Recall@20 and 3.8% in NDCG@20 over state-of-the-art approaches, validating the effectiveness of noise-robust modeling and fine-grained semantic alignment in multimodal recommendation.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Multimodal recommendation systems integrate diverse multimodal information into the feature representations of both items and users, thereby enabling a more comprehensive modeling of user preferences. However, existing methods are hindered by data sparsity and the inherent noise within multimodal data, which impedes the accurate capture of users' interest preferences. Additionally, discrepancies in the semantic representations of items across different modalities can adversely impact the prediction accuracy of recommendation models. To address these challenges, we introduce a novel diffusion-based contrastive learning framework (DiffCL) for multimodal recommendation. DiffCL employs a diffusion model to generate contrastive views that effectively mitigate the impact of noise during the contrastive learning phase. Furthermore, it improves semantic consistency across modalities by aligning distinct visual and textual semantic information through stable ID embeddings. Finally, the introduction of the Item-Item Graph enhances multimodal feature representations, thereby alleviating the adverse effects of data sparsity on the overall system performance. We conduct extensive experiments on three public datasets, and the results demonstrate the superiority and effectiveness of the DiffCL.

Problem

• Multimodal Recommendation Systems
• Data Sparsity
• Semantic Heterogeneity

Innovation

• DiffCL
• Multimodal Recommendation
• Diffusion Models