RecPS: Privacy Risk Scoring for Recommender Systems

2025-07-24

AI Summary

This paper addresses the challenge of quantifying fine-grained privacy risks in user-item interaction data within recommender systems. To this end, it proposes RecPS—a novel framework that introduces, for the first time, a differential privacy–inspired definition of interaction-level privacy scores, extended to the user level, thereby establishing an interpretable and computationally tractable privacy risk assessment system. Its core innovation is RecLiRA, a method leveraging membership inference attacks (MIA) to achieve high-accuracy estimation of interaction-level membership probabilities, dynamically generating privacy scores during model training while maintaining compatibility with mainstream recommendation models and datasets. RecPS further enables privacy-risk visualization and selective data forgetting. Extensive experiments on multiple benchmark datasets demonstrate its superiority over existing approaches, effectively identifying high-risk interactions. The framework establishes a new paradigm for privacy-aware modeling and compliance-driven data governance.

Technology Category

• Machine Learning: Privacy
• Data Mining & Knowledge Management: Recommender Systems
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Recommender systems (RecSys) have become an essential component of many web applications. The core of the system is a recommendation model trained on highly sensitive user-item interaction data. While privacy-enhancing techniques are actively studied in the research community, the real-world model development still depends on minimal privacy protection, e.g., via controlled access. Users of such systems should have the right to choose emph{not} to share highly sensitive interactions. However, there is no method allowing the user to know which interactions are more sensitive than others. Thus, quantifying the privacy risk of RecSys training data is a critical step to enabling privacy-aware RecSys model development and deployment. We propose a membership-inference attack (MIA)- based privacy scoring method, RecPS, to measure privacy risks at both the interaction and user levels. The RecPS interaction-level score definition is motivated and derived from differential privacy, which is then extended to the user-level scoring method. A critical component is the interaction-level MIA method RecLiRA, which gives high-quality membership estimation. We have conducted extensive experiments on well-known benchmark datasets and RecSys models to show the unique features and benefits of RecPS scoring in risk assessment and RecSys model unlearning. Our code is available at https://anonymous.4open.science/r/RsLiRA-4BD3/readme.md.

Problem

• Quantify privacy risks in recommender system training data
• Measure sensitivity of user-item interactions for privacy awareness
• Develop scoring method for interaction and user-level privacy risks

Innovation

• MIA-based privacy scoring for RecSys
• Differential privacy-derived interaction scoring
• User-level privacy risk quantification