Accurate and Diverse Recommendations via Propensity-Weighted Linear Autoencoders

2025-12-23

AI Summary

In real-world recommender systems, user-item interactions exhibit Missing-Not-At-Random (MNAR) patterns, leading to over-observation of popular items and systematic under-observation of long-tail items—thereby inducing selection bias and degrading recommendation diversity. Conventional Inverse Propensity Scoring (IPS) methods, which assume power-law propensity distributions, tend to over-penalize popular items, harming recommendation accuracy. To address this, we propose Log-Sigmoid Propensity Modeling, replacing the power-law assumption with a log-sigmoid function to more faithfully characterize item observation probabilities and mitigate over-correction for popular items. We integrate this modeling into a lightweight linear autoencoder framework, enabling interpretable and computationally efficient debiasing. Extensive experiments on multiple benchmark datasets demonstrate that our approach significantly improves diversity (ILS ↑18.7%) while maintaining—or slightly improving—accuracy (Recall@20 ↑0.9%), achieving a balanced optimization of both objectives.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Recommender Systems
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

In real-world recommender systems, user-item interactions are Missing Not At Random (MNAR), as interactions with popular items are more frequently observed than those with less popular ones. Missing observations shift recommendations toward frequently interacted items, which reduces the diversity of the recommendation list. To alleviate this problem, Inverse Propensity Scoring (IPS) is widely used and commonly models propensities based on a power-law function of item interaction frequency. However, we found that such power-law-based correction overly penalizes popular items and harms their recommendation performance. We address this issue by redefining the propensity score to allow broader item recommendation without excessively penalizing popular items. The proposed score is formulated by applying a sigmoid function to the logarithm of the item observation frequency, maintaining the simplicity of power-law scoring while allowing for more flexible adjustment. Furthermore, we incorporate the redefined propensity score into a linear autoencoder model, which tends to favor popular items, and evaluate its effectiveness. Experimental results revealed that our method substantially improves the diversity of items in the recommendation list without sacrificing recommendation accuracy.

Problem

• Addresses MNAR bias in user-item interactions
• Redefines propensity scoring to balance popularity and diversity
• Enhances recommendation diversity without compromising accuracy

Innovation

• Sigmoid-based propensity scoring for flexible adjustment
• Linear autoencoder integration to balance popularity bias
• Improved diversity without sacrificing recommendation accuracy