From Generation to Consumption: Personalized List Value Estimation for Re-ranking

2025-08-04

AI Summary

In recommendation system re-ranking, existing methods often neglect user early-exit behavior, leading to a mismatch between the generated list’s estimated value and its actual consumption value. To address this, we propose CAVE (Context-Aware Value Estimation), a personalized list-value estimation framework that explicitly models position-dependent exit probability for the first time. CAVE defines list value as the expectation of sub-list values, disentangling exit causes into interest-driven and stochastic fatigue components. The fatigue component is modeled via a Weibull distribution, enabling joint optimization with sub-list value estimation. Evaluated on three major online leaderboards at Kuaishou, the Amazon dataset, and large-scale A/B tests, CAVE consistently outperforms strong baselines—demonstrating significant improvements in both value estimation fidelity and downstream recommendation performance.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Learning Human Values and Preferences
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Re-ranking is critical in recommender systems for optimizing the order of recommendation lists, thus improving user satisfaction and platform revenue. Most existing methods follow a generator-evaluator paradigm, where the evaluator estimates the overall value of each candidate list. However, they often ignore the fact that users may exit before consuming the full list, leading to a mismatch between estimated generation value and actual consumption value. To bridge this gap, we propose CAVE, a personalized Consumption-Aware list Value Estimation framework. CAVE formulates the list value as the expectation over sub-list values, weighted by user-specific exit probabilities at each position. The exit probability is decomposed into an interest-driven component and a stochastic component, the latter modeled via a Weibull distribution to capture random external factors such as fatigue. By jointly modeling sub-list values and user exit behavior, CAVE yields a more faithful estimate of actual list consumption value. We further contribute three large-scale real-world list-wise benchmarks from the Kuaishou platform, varying in size and user activity patterns. Extensive experiments on these benchmarks, two Amazon datasets, and online A/B testing on Kuaishou show that CAVE consistently outperforms strong baselines, highlighting the benefit of explicitly modeling user exits in re-ranking.

Problem

• Estimating personalized list value considering user exit behavior
• Bridging gap between generation and actual consumption value
• Modeling user exit probabilities with interest and stochastic factors

Innovation

• Personalized list value estimation framework
• User exit probability modeling via Weibull distribution
• Joint sub-list value and exit behavior modeling