UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation

2026-06-25

AI Summary

This work addresses the limitations of existing industrial recommendation systems, which predominantly rely on component-level model scaling and lack a unified framework for holistic model scaling that balances efficiency and performance. To bridge this gap, the authors propose UniFormer, the first unified scaling framework that spans the entire modeling space. UniFormer decouples the modeling space into feature and task subspaces, employing stacked interaction modules to model each separately. It further introduces semantic tokenization to decouple user-item representations, accelerating inference. Additionally, the framework integrates multi-sequence cross-attention with multi-view feed-forward networks (FFNs) to enhance representational capacity. Deployed in both Kuaishou and Kuaishou Lite, UniFormer significantly improves user engagement metrics, increasing app dwell time by 0.101% and 0.260%, and watch time by 0.729% and 1.113%, respectively.

Abstract

Recently, substantial progress has been made in industrial recommendation through component-centric model scaling, where individual components such as behavior modeling, feature interaction, or task modeling are independently scaled to improve model capacity. Although recent methods such as HyFormer and OneTrans further explore cross-module co-scaling by jointly modeling behavior and interaction, their designs are still confined to the feature space and lack a unified model-centric scaling framework over the overall modeling space. In this paper, we propose UniFormer, an efficient and unified model-centric scaling framework for industrial recommender systems. To improve efficiency, UniFormer decomposes the overall modeling space into feature and task spaces, which are modeled by stacked Feature-space Interaction Modules and Task-space Interaction Modules, respectively. Moreover, UniFormer introduces semantic-based tokenization scheme to enable user-item decoupling, thereby achieving request-level inference acceleration. To prevent preference collapse, UniFormer employs multi-sequence cross-attention to separately capture heterogeneous behavior patterns, followed by the self-attention to enhance interaction modeling. Besides, dedicated multi-view FFNs are introduced to support flexible and scalable parameter scaling across different modeling components. Extensive online A/B testing in two production scenarios, Kuaishou and Kuaishou Lite, shows that UniFormer consistently improves user engagement and interaction metrics, achieving gains of +0.101%/+0.260% in App Stay Time and +0.729%/+1.113% in Watch Time, respectively.

Problem

• model-centric scaling
• industrial recommendation
• unified framework
• feature interaction
• behavior modeling

Innovation

• model-centric scaling
• semantic tokenization
• user-item decoupling
• multi-sequence cross-attention
• multi-view FFN