Privacy-Preserving Multimodal News Recommendation through Federated Learning

2025-07-21

AI Summary

Traditional personalized news recommendation systems face three key challenges: excessive reliance on textual features, neglect of users’ short-term interests, and privacy leakage due to centralized data storage. To address these, we propose a privacy-preserving multimodal federated news recommendation framework. First, it constructs multimodal news representations by jointly encoding textual and visual features. Second, it introduces a time-aware dual-timescale interest modeling mechanism to dynamically capture both long- and short-term user interest evolution. Third, within a federated learning paradigm, it employs Shamir’s secret sharing for secure model aggregation, eliminating the need to upload raw user data. The framework achieves a favorable trade-off between recommendation accuracy and privacy protection. Extensive experiments on real-world datasets demonstrate significant improvements over state-of-the-art baselines—up to 3.2%–5.7% gains in Recall@5 and NDCG@5—while satisfying rigorous security requirements for aggregation.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Personalized News Recommendation systems (PNR) have emerged as a solution to information overload by predicting and suggesting news items tailored to individual user interests. However, traditional PNR systems face several challenges, including an overreliance on textual content, common neglect of short-term user interests, and significant privacy concerns due to centralized data storage. This paper addresses these issues by introducing a novel multimodal federated learning-based approach for news recommendation. First, it integrates both textual and visual features of news items using a multimodal model, enabling a more comprehensive representation of content. Second, it employs a time-aware model that balances users' long-term and short-term interests through multi-head self-attention networks, improving recommendation accuracy. Finally, to enhance privacy, a federated learning framework is implemented, enabling collaborative model training without sharing user data. The framework divides the recommendation model into a large server-maintained news model and a lightweight user model shared between the server and clients. The client requests news representations (vectors) and a user model from the central server, then computes gradients with user local data, and finally sends their locally computed gradients to the server for aggregation. The central server aggregates gradients to update the global user model and news model. The updated news model is further used to infer news representation by the server. To further safeguard user privacy, a secure aggregation algorithm based on Shamir's secret sharing is employed. Experiments on a real-world news dataset demonstrate strong performance compared to existing systems, representing a significant advancement in privacy-preserving personalized news recommendation.

Problem

• Overcoming reliance on text-only content in news recommendation
• Balancing long-term and short-term user interests effectively
• Ensuring privacy via federated learning without data sharing

Innovation

• Multimodal model integrates text and visual features
• Time-aware model balances long and short-term interests
• Federated learning with secure aggregation ensures privacy