DUET: Dual Model Co-Training for Entire Space CTR Prediction

2025-10-28

AI Summary

Pre-ranking faces an inherent trade-off between model expressiveness and computational efficiency: while dual-tower architectures offer low latency, they fail to capture collaborative or inhibitory relationships among candidate items, resulting in poor contextual coherence, limited diversity, and exacerbated sampling bias. To address this, we propose a set-wise prediction framework that jointly scores subsets of candidates in a single forward pass, enabling inter-candidate modeling via an information interaction module and amortizing computation. We further introduce a novel dual-model co-training mechanism, where models mutually generate pseudo-labels for unexposed samples to strengthen supervision and mitigate distribution shift. Our approach preserves the low-latency advantage of dual-tower architectures while significantly enhancing expressiveness. Deployed at scale across Kuaishou’s main app and Lite version, it serves hundreds of millions of users and consistently outperforms state-of-the-art methods in CTR, diversity, and key business metrics.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The pre-ranking stage plays a pivotal role in large-scale recommender systems but faces an intrinsic trade-off between model expressiveness and computational efficiency. Owing to the massive candidate pool and strict latency constraints, industry systems often rely on lightweight two-tower architectures, which are computationally efficient yet limited in estimation capability. As a result, they struggle to capture the complex synergistic and suppressive relationships among candidate items, which are essential for producing contextually coherent and diverse recommendation lists. Moreover, this simplicity further amplifies the Sample Selection Bias (SSB) problem, as coarse-grained models trained on biased exposure data must generalize to a much larger candidate space with distinct distributions. To address these issues, we propose extbf{DUET} ( extbf{DU}al Model Co-Training for extbf{E}ntire Space C extbf{T}R Prediction), a set-wise pre-ranking framework that achieves expressive modeling under tight computational budgets. Instead of scoring items independently, DUET performs set-level prediction over the entire candidate subset in a single forward pass, enabling information-aware interactions among candidates while amortizing the computational cost across the set. Moreover, a dual model co-training mechanism extends supervision to unexposed items via mutual pseudo-label refinement, effectively mitigating SSB. Validated through extensive offline experiments and online A/B testing, DUET consistently outperforms state-of-the-art baselines and achieves improvements across multiple core business metrics. At present, DUET has been fully deployed in Kuaishou and Kuaishou Lite Apps, serving the main traffic for hundreds of millions of users.

Problem

• Addresses trade-off between model expressiveness and computational efficiency
• Captures complex synergistic relationships among candidate items
• Mitigates Sample Selection Bias in large-scale recommender systems

Innovation

• Set-level prediction over entire candidate subsets
• Dual model co-training with mutual pseudo-label refinement
• Amortizes computational cost across candidate sets