Balancing Accuracy and Novelty with Sub-Item Popularity

2025-08-07

AI Summary

In music recommendation, personalized popularity scoring (PPS) tends to reinforce users’ familiarity bias, undermining novelty and consequently degrading long-term user engagement. To address this, we propose sub-item ID-level personalized popularity scoring (sPPS), which models fine-grained repetition preferences within a Transformer-based architecture—thereby overcoming the limitation of conventional item-level PPS in capturing shared repetition patterns. Leveraging the RecJPQ sub-item decomposition framework, sPPS enables explicit accuracy–novelty trade-offs. Extensive experiments demonstrate that sPPS significantly outperforms strong baselines across multiple metrics: it substantially improves personalized novelty while preserving—and in many cases enhancing—recommendation accuracy. The implementation is publicly available.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Recommender Systems
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

In the realm of music recommendation, sequential recommenders have shown promise in capturing the dynamic nature of music consumption. A key characteristic of this domain is repetitive listening, where users frequently replay familiar tracks. To capture these repetition patterns, recent research has introduced Personalised Popularity Scores (PPS), which quantify user-specific preferences based on historical frequency. While PPS enhances relevance in recommendation, it often reinforces already-known content, limiting the system's ability to surface novel or serendipitous items - key elements for fostering long-term user engagement and satisfaction. To address this limitation, we build upon RecJPQ, a Transformer-based framework initially developed to improve scalability in large-item catalogues through sub-item decomposition. We repurpose RecJPQ's sub-item architecture to model personalised popularity at a finer granularity. This allows us to capture shared repetition patterns across sub-embeddings - latent structures not accessible through item-level popularity alone. We propose a novel integration of sub-ID-level personalised popularity within the RecJPQ framework, enabling explicit control over the trade-off between accuracy and personalised novelty. Our sub-ID-level PPS method (sPPS) consistently outperforms item-level PPS by achieving significantly higher personalised novelty without compromising recommendation accuracy. Code and experiments are publicly available at https://github.com/sisinflab/Sub-id-Popularity.

Problem

• Balancing accuracy and novelty in music recommendations
• Addressing repetitive listening with personalized popularity scores
• Enhancing recommendation novelty without sacrificing accuracy

Innovation

• Sub-item decomposition for fine-grained popularity modeling
• Transformer-based RecJPQ framework for scalability
• Sub-ID-level PPS for balancing accuracy and novelty