From Pairwise to Ranking: Climbing the Ladder to Ideal Collaborative Filtering with Pseudo-Ranking

2024-12-24

AI Summary

Collaborative filtering suffers from limited performance due to the absence of complete user preference rankings over all items, forcing reliance on pairwise loss functions as crude surrogates for true ranking optimization. To address this, we propose the Pseudo-Ranking Paradigm (PRP): it generates high-quality pseudo-full-order labels via controlled noise injection and introduces a gradient-aware confidence-weighted ranking loss, enabling end-to-end optimization of the actual ranking objective for the first time. PRP theoretically bridges the gap between conventional pairwise learning and ideal full-order learning, while preserving differentiability and robustness. Extensive experiments on four public benchmark datasets demonstrate an average 12.7% improvement in top-K recommendation accuracy over state-of-the-art methods. Moreover, PRP exhibits strong robustness against noise in pseudo-labels, confirming its practical reliability and generalizability.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Sampling/Simulation-based Search
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Intuitively, an ideal collaborative filtering (CF) model should learn from users' full rankings over all items to make optimal top-K recommendations. Due to the absence of such full rankings in practice, most CF models rely on pairwise loss functions to approximate full rankings, resulting in an immense performance gap. In this paper, we provide a novel analysis using the multiple ordinal classification concept to reveal the inevitable gap between a pairwise approximation and the ideal case. However, bridging the gap in practice encounters two formidable challenges: (1) none of the real-world datasets contains full ranking information; (2) there does not exist a loss function that is capable of consuming ranking information. To overcome these challenges, we propose a pseudo-ranking paradigm (PRP) that addresses the lack of ranking information by introducing pseudo-rankings supervised by an original noise injection mechanism. Additionally, we put forward a new ranking loss function designed to handle ranking information effectively. To ensure our method's robustness against potential inaccuracies in pseudo-rankings, we equip the ranking loss function with a gradient-based confidence mechanism to detect and mitigate abnormal gradients. Extensive experiments on four real-world datasets demonstrate that PRP significantly outperforms state-of-the-art methods.

Problem

• Bridging performance gap between pairwise loss and ideal full rankings
• Addressing absence of real-world full ranking information in datasets
• Designing effective ranking loss function with confidence mechanism

Innovation

• Pseudo-ranking paradigm with noise injection
• New ranking loss function design
• Gradient-based confidence mechanism