Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation

2025-09-30

AI Summary

Recommendation systems face challenges including extreme sparsity of user behavioral data, mismatch between continuous noise modeling and discrete user preferences, and difficulties in capturing nuanced preference intensities. To address these, we propose PreferGrow—the first discrete diffusion-based recommendation framework. Its core contributions are: (1) introducing a learnable “preference ratio” as a discrete state variable to explicitly model relative preference strength; (2) replacing conventional Gaussian noise injection with controllable preference decay, eliminating restrictive prior assumptions; and (3) designing an iterative signal growth mechanism that progressively reconstructs preference structures via a Markovian and invertible matrixized process. PreferGrow integrates negative-sampling-inspired perturbation strategies with discrete diffusion modeling. Extensive experiments on five benchmark datasets demonstrate that PreferGrow significantly outperforms existing diffusion-based recommendation models, achieving superior robustness and consistent performance gains—particularly under severe data sparsity.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Recommenders aim to rank items from a discrete item corpus in line with user interests, yet suffer from extremely sparse user preference data. Recent advances in diffusion models have inspired diffusion-based recommenders, which alleviate sparsity by injecting noise during a forward process to prevent the collapse of perturbed preference distributions. However, current diffusion-based recommenders predominantly rely on continuous Gaussian noise, which is intrinsically mismatched with the discrete nature of user preference data in recommendation. In this paper, building upon recent advances in discrete diffusion, we propose PreferGrow, a discrete diffusion-based recommender system that models preference ratios by fading and growing user preferences over the discrete item corpus. PreferGrow differs from existing diffusion-based recommenders in three core aspects: (1) Discrete modeling of preference ratios: PreferGrow models relative preference ratios between item pairs, rather than operating in the item representation or raw score simplex. This formulation aligns naturally with the discrete and ranking-oriented nature of recommendation tasks. (2) Perturbing via preference fading: Instead of injecting continuous noise, PreferGrow fades user preferences by replacing the preferred item with alternatives -- physically akin to negative sampling -- thereby eliminating the need for any prior noise assumption. (3) Preference reconstruction via growing: PreferGrow reconstructs user preferences by iteratively growing the preference signals from the estimated ratios. PreferGrow offers a well-defined matrix-based formulation with theoretical guarantees on Markovianity and reversibility, and it demonstrates consistent performance gains over state-of-the-art diffusion-based recommenders across five benchmark datasets, highlighting both its theoretical soundness and empirical effectiveness.

Problem

• Modeling discrete preference ratios for recommendation tasks
• Addressing data sparsity via discrete diffusion without noise assumptions
• Reconstructing user preferences through iterative preference signal growth

Innovation

• Models preference ratios via discrete diffusion
• Fades preferences by replacing items without noise
• Reconstructs preferences by growing signals from ratios