USM: Unbiased Survey Modeling for Limiting Negative User Experiences in Recommendation Systems

2024-12-14

arXiv.org

AI Summary

This paper addresses the problem of excessive exposure to negative content in recommender systems, caused by sparse negative feedback. To mitigate this, we propose an unbiased user-survey-based framework for modeling and correcting negative feedback. Methodologically, we (1) design a lightweight online mechanism for unbiased survey distribution; (2) develop a survey-response modeling architecture that integrates latent hidden unit contribution (LHUC) learning with Squeeze-and-Excitation attention; and (3) introduce a survey-submit bias correction module to alleviate response biases induced by user behavior. Deployed online, our framework reduces report rate, aversion rate, and inappropriate survey rate by 1.75%, 2.57%, and 2.06%, respectively. A/B testing further demonstrates reductions of 1.44%–3.9% in survey-triggered content rate and inappropriate content rate, and 1.0%–2.27% in report and aversion rates—significantly enhancing user experience and content safety.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Reducing negative user experiences is essential for the success of recommendation platforms. Exposing users to inappropriate content could not only adversely affect users' psychological well-beings, but also potentially drive users away from the platform, sabotaging the platform's long-term success. However, recommendation algorithms tend to weigh more heavily on positive feedback signals due to the scarcity of negative ones, which may result in the neglect of valuable negative user feedback. In this paper, we propose an approach aimed at limiting negative user experiences. Our method primarily relies on distributing in-feed surveys to the users, modeling the users' feedback collected from the survey, and integrating the model predictions into the recommendation system. We further enhance the baseline survey model by integrating the Learning Hidden Unit Contributions module and the Squeeze-and-Excitation module. In addition, we strive to resolve the problem of response Bias by applying a survey-submit model; The A/B testing results indicate a reduction in survey sexual rate and survey inappropriate rate, ranging from -1.44% to -3.9%. Additionally, we compared our methods against an online baseline that does not incorporate our approach. The results indicate that our approach significantly reduces the report rate and dislike rate by 1% to 2.27% compared to the baseline, confirming the effectiveness of our methods in enhancing user experience. After we launched the survey model based our approach on our platform, the model is able to bring reductions of 1.75%, 2.57%, 2.06% on reports, dislikes, survey inappropriate rate, respectively.

Problem

• Limiting negative user experiences
• Reducing inappropriate content exposure
• Improving recommendation system feedback integration

Innovation

• Distributes in-feed surveys
• Integrates Learning Hidden Unit Contributions
• Applies survey-submit model