How to Diversify any Personalized Recommender?

2024-05-03

AI Summary

To address the longstanding trade-off between diversity and accuracy in recommender systems, this paper proposes a user-centric interactive preprocessing framework. It introduces controllable, category-aware distribution shifts via dynamic interaction sampling and reweighting grounded in user profiles—enhancing recommendation diversity without altering the underlying model architecture. The method is plug-and-play, compatible with arbitrary recommendation models, and simultaneously improves provider fairness. Experiments on news and book datasets demonstrate a 27.3% improvement in Top-N diversity, a 41.8% increase in long-tail category exposure, and stable or slightly improved accuracy—across both traditional and neural recommender models. The core innovation lies in the first integration of explicit distribution shift control into user-level interaction preprocessing, thereby unifying diversity, accuracy, and fairness optimization within a single, lightweight, model-agnostic framework.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

In this paper, we introduce a novel approach to improve the diversity of Top-N recommendations while maintaining accuracy. Our approach employs a user-centric pre-processing strategy aimed at exposing users to a wide array of content categories and topics. We personalize this strategy by selectively adding and removing a percentage of interactions from user profiles. This personalization ensures we remain closely aligned with user preferences while gradually introducing distribution shifts. Our pre-processing technique offers flexibility and can seamlessly integrate into any recommender architecture. We run extensive experiments on two publicly available data sets for news and book recommendations to evaluate our approach. We test various standard and neural network-based recommender system algorithms. Our results show that our approach generates diverse recommendations, ensuring users are exposed to a wider range of items. Furthermore, using pre-processed data for training leads to recommender systems achieving performance levels comparable to, and in some cases, better than those trained on original, unmodified data. Additionally, our approach promotes provider fairness by facilitating exposure to minority categories. Our GitHub code is available at: https://github.com/SlokomManel/How-to-Diversify-any-Personalized-Recommender-

Problem

• Improve diversity in Top-N recommendations
• Maintain accuracy with user-centric strategy
• Enhance provider fairness through minority exposure

Innovation

• User-centric pre-processing strategy
• Selective interaction modification
• Seamless integration into recommenders