Adaptive Quality-Diversity Trade-offs for Large-Scale Batch Recommendation

2026-02-02

AI Summary

This work addresses the critical challenge in large-scale recommender systems of simultaneously maintaining recommendation quality, effectively controlling diversity, mitigating user churn, and managing computational complexity with million-scale item catalogs. The authors propose B-DivRec, a novel algorithm that uniquely integrates Determinantal Point Processes (DPPs) with a fuzzy de-redundancy mechanism to efficiently balance quality and diversity within recommendation batches. Furthermore, an adaptive online learning strategy is introduced to dynamically adjust the strength of diversity based on real-time user feedback. Extensive experiments on both synthetic and real-world datasets—spanning movie recommendation and drug repositioning tasks—demonstrate the method’s effectiveness: B-DivRec significantly enhances the synergy between recommendation diversity and positive user engagement while preserving scalability.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

A core research question in recommender systems is to propose batches of highly relevant and diverse items, that is, items personalized to the user's preferences, but which also might get the user out of their comfort zone. This diversity might induce properties of serendipidity and novelty which might increase user engagement or revenue. However, many real-life problems arise in that case: e.g., avoiding to recommend distinct but too similar items to reduce the churn risk, and computational cost for large item libraries, up to millions of items. First, we consider the case when the user feedback model is perfectly observed and known in advance, and introduce an efficient algorithm called B-DivRec combining determinantal point processes and a fuzzy denuding procedure to adjust the degree of item diversity. This helps enforcing a quality-diversity trade-off throughout the user history. Second, we propose an approach to adaptively tailor the quality-diversity trade-off to the user, so that diversity in recommendations can be enhanced if it leads to positive feedback, and vice-versa. Finally, we illustrate the performance and versatility of B-DivRec in the two settings on synthetic and real-life data sets on movie recommendation and drug repurposing.

Problem

• recommendation
• diversity
• quality-diversity trade-off
• large-scale
• batch recommendation

Innovation

• Quality-Diversity Trade-off
• Determinantal Point Processes
• Adaptive Recommendation
• Batch Recommendation
• Fuzzy Denuding