AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking

2025-11-16

AI Summary

To address redundant computation and high latency caused by serial DNN execution in industrial recommendation pre-ranking, this paper proposes the Asynchronous Inference Framework (AIF). AIF’s core innovation is the first decoupling of user-side and item-side non-interaction computations into nearline and parallel stages: it leverages feature precomputation, computation reuse, and lightweight approximate modeling of interaction modules to achieve low-latency, high-throughput real-time inference. Adopting a co-design of framework and model, AIF is deployed and validated in Taobao’s display advertising system. Results show a significant reduction in pre-ranking latency, over 30% savings in computational resources, support for more sophisticated models and higher-dimensional features, and a substantial improvement in online CTR.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

In industrial recommendation systems, pre-ranking models based on deep neural networks (DNNs) commonly adopt a sequential execution framework: feature fetching and model forward computation are triggered only after receiving candidates from the upstream retrieval stage. This design introduces inherent bottlenecks, including redundant computations of identical users/items and increased latency due to strictly sequential operations, which jointly constrain the model's capacity and system efficiency. To address these limitations, we propose the Asynchronous Inference Framework (AIF), a cost-effective computational architecture that decouples interaction-independent components, those operating within a single user or item, from real-time prediction. AIF reorganizes the model inference process by performing user-side computations in parallel with the retrieval stage and conducting item-side computations in a nearline manner. This means that interaction-independent components are calculated just once and completed before the real-time prediction phase of the pre-ranking stage. As a result, AIF enhances computational efficiency and reduces latency, freeing up resources to significantly improve the feature set and model architecture of interaction-independent components. Moreover, we delve into model design within the AIF framework, employing approximated methods for interaction-dependent components in online real-time predictions. By co-designing both the framework and the model, our solution achieves notable performance gains without significantly increasing computational and latency costs. This has enabled the successful deployment of AIF in the Taobao display advertising system.

Problem

• Reduces redundant computations in pre-ranking models
• Decouples interaction-independent components from real-time prediction
• Minimizes latency through asynchronous parallel computation

Innovation

• Decouples user-item independent components from real-time prediction
• Performs user computations parallel to retrieval stage
• Uses approximated methods for interaction-dependent online components