Scaling Laws for Online Advertisement Retrieval

2024-11-20

arXiv.org

AI Summary

Existing online advertising retrieval systems lack generalizable scaling laws due to prohibitively high online experimentation costs and heterogeneous cross-scenario configurations. Method: This work establishes, for the first time in a real-world industrial system (Kuaishou Advertising), a quantitative scaling relationship between revenue and computational cost (FLOPs). We propose a lightweight modeling paradigm that unifies machine cost and online revenue via FLOPs as the sole scaling factor, shifting expensive online evaluation to efficient offline validation. To overcome cross-scenario generalization bottlenecks, we design a highly correlated proxy metric, R/R*. We further develop an end-to-end framework encompassing FLOPs sensitivity analysis, cost simulation, scaling law fitting, and empirical validation. Contribution/Results: Empirical results confirm strong adherence to the derived scaling law, enabling ROI-constrained model design and dynamic multi-scenario resource allocation. The approach significantly improves revenue per unit of computation, demonstrating both theoretical validity and practical impact.

Technology Category

• Machine Learning: Scalability of ML Systems
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

The scaling law is a notable property of neural network models and has significantly propelled the development of large language models. Scaling laws hold great promise in guiding model design and resource allocation. Recent research increasingly shows that scaling laws are not limited to NLP tasks or Transformer architectures; they also apply to domains such as recommendation. However, there is still a lack of literature on scaling law research in online advertisement retrieval systems. This may be because 1) identifying the scaling law for resource cost and online revenue is often expensive in both time and training resources for large-scale industrial applications, and 2) varying settings for different systems prevent the scaling law from being applied across various scenarios. To address these issues, we propose a lightweight paradigm to identify the scaling law of online revenue and machine cost for a certain online advertisement retrieval scenario with a low experimental cost. Specifically, we focus on a sole factor (FLOPs) and propose an offline metric named R/R* that exhibits a high linear correlation with online revenue for retrieval models. We estimate the machine cost offline via a simulation algorithm. Thus, we can transform most online experiments into low-cost offline experiments. We conduct comprehensive experiments to verify the effectiveness of our proposed metric R/R* and to identify the scaling law in the online advertisement retrieval system of Kuaishou. With the scaling law, we demonstrate practical applications for ROI-constrained model designing and multi-scenario resource allocation in Kuaishou advertising system. To the best of our knowledge, this is the first work to study the scaling laws for online advertisement retrieval of real-world systems, showing great potential for scaling law in advertising system optimization.

Problem

• Lack of scaling law research in online advertisement retrieval systems
• High cost of identifying scaling laws for industrial applications
• Difficulty applying scaling laws across varying system settings

Innovation

• Lightweight paradigm identifies online scaling laws
• Novel offline metric correlates with online revenue
• Offline simulation algorithm estimates machine costs