MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever

2025-08-27

AI Summary

In industrial recommendation systems, a semantic gap exists between single-objective modeling in the retrieval stage and multi-objective optimization in the ranking stage; conventional parallel multi-path architectures incur linear resource overhead and struggle to model loosely coupled objectives. This paper proposes the Dynamic Multi-Task Transformer (DMT), the first framework enabling end-to-end multi-objective joint modeling in retrieval. DMT unifies user behavior sequences and multi-objective semantics via target-conditioned attention, personalized task-weight learning, and user-aware token representation and network architecture. Deployed in Kuaishou’s short-video recommendation system serving over 400 million DAUs, DMT significantly improves both click-through rate and watch time. Compared to parallel multi-path baselines, it achieves superior performance while reducing GPU resource consumption by 37%.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Computer Vision: Multi-modal Vision
• Machine Learning: Learning Preferences or Rankings

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Modern industrial recommendation systems encounter a core challenge of multi-stage optimization misalignment: a significant semantic gap exists between the multi-objective optimization paradigm widely used in the ranking phase and the single-objective modeling in the retrieve phase. Although the mainstream industry solution achieves multi-objective coverage through parallel multi-path single-objective retrieval, this approach leads to linear growth of training and serving resources with the number of objectives and has inherent limitations in handling loosely coupled objectives. This paper proposes the MPFormer, a dynamic multi-task Transformer framework, which systematically addresses the aforementioned issues through three innovative mechanisms. First, an objective-conditioned transformer that jointly encodes user behavior sequences and multi-task semantics through learnable attention modulation; second, personalized target weights are introduced to achieve dynamic adjustment of retrieval results; finally, user personalization information is incorporated into token representations and the Transformer structure to further enhance the model's representation ability. This framework has been successfully integrated into Kuaishou short video recommendation system, stably serving over 400 million daily active users. It significantly improves user daily engagement and system operational efficiency. Practical deployment verification shows that, compared with traditional solutions, it effectively optimizes the iterative paradigm of multi-objective retrieval while maintaining service response speed, providing a scalable multi-objective solution for industrial recommendation systems.

Problem

• Addresses multi-stage optimization misalignment in recommendation systems
• Solves linear resource growth with increasing objectives in retrieval
• Handles loosely coupled objectives in multi-task retrieval frameworks

Innovation

• Dynamic multi-task Transformer for retrieval
• Objective-conditioned attention modulation mechanism
• Personalized token representations with adaptive weights