Generative Regression Based Watch Time Prediction for Video Recommendation: Model and Performance

2024-12-28

AI Summary

In short-video recommendation, predicting users’ watch time (WTP) suffers from regression bias due to large temporal span and highly skewed duration distributions. This paper proposes Generative Regression (GR), a novel paradigm that reformulates continuous duration prediction as a numerical-to-sequence generation task. GR employs structured discrete encoding to enable lossless reconstruction and high-fidelity prediction. We innovatively design a curriculum learning strategy, embedding-level Mixup augmentation, and a Teacher-Forcing optimization mechanism to mitigate train-inference inconsistency. Extensive experiments on four public benchmarks and Kuaishou’s industrial dataset demonstrate significant improvements over state-of-the-art methods. Online A/B testing shows measurable gains in average watch time. Furthermore, GR successfully transfers to lifetime value (LTV) prediction, validating its generalizability across sequential regression tasks in recommender systems.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Watch time prediction (WTP) has emerged as a pivotal task in short video recommendation systems, designed to encapsulate user interests. Predicting users' watch times on videos often encounters challenges, including wide value ranges and imbalanced data distributions, which can lead to significant bias when directly regressing watch time. Recent studies have tried to tackle these issues by converting the continuous watch time estimation into an ordinal classification task. While these methods are somewhat effective, they exhibit notable limitations. Inspired by language modeling, we propose a novel Generative Regression (GR) paradigm for WTP based on sequence generation. This approach employs structural discretization to enable the lossless reconstruction of original values while maintaining prediction fidelity. By formulating the prediction problem as a numerical-to-sequence mapping, and with meticulously designed vocabulary and label encodings, each watch time is transformed into a sequence of tokens. To expedite model training, we introduce the curriculum learning with an embedding mixup strategy which can mitigate training-and-inference inconsistency associated with teacher forcing. We evaluate our method against state-of-the-art approaches on four public datasets and one industrial dataset. We also perform online A/B testing on Kuaishou, a leading video app with about 400 million DAUs, to demonstrate the real-world efficacy of our method. The results conclusively show that GR outperforms existing techniques significantly. Furthermore, we successfully apply GR to another regression task in recommendation systems, i.e., Lifetime Value (LTV) prediction, which highlights its potential as a novel and effective solution to general regression challenges.

Problem

• Short Video Recommendation
• Watching Time Prediction
• Data Imbalance

Innovation

• Generative Regression (GR)
• Sequence Generation Task
• Hybrid Learning Strategy