BAT: Benchmark for Auto-bidding Task

2025-04-22

The Web Conference

AI Summary

Standardized benchmarks for automated bidding in online advertising are lacking, hindering algorithm development and fair evaluation. This paper introduces the first programmatically oriented autobidding benchmark supporting two auction mechanisms—English auctions and generalized second-price (GSP) auctions. It integrates a realistic log-driven synthetic-augmented dataset, an end-to-end evaluation protocol, and a modular open-source framework. The benchmark unifies support for core real-time bidding (RTB) objectives—including budget pacing and cost-per-click (CPC) constraint optimization—and accommodates diverse methodological paradigms: reinforcement learning, control-theoretic approaches, and optimization-based baselines. We conduct a systematic evaluation of 12 state-of-the-art autobidding methods, demonstrating significant performance differences across key metrics: budget deviation (<5%), CPC compliance rate (>92%), and ROI stability. The codebase and dataset are publicly released and have been widely adopted by the research community.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Sampling/Simulation-based Search
• Game Theory and Economic Paradigms: Auctions and Market-Based Systems

Application Category

• Economics, Online Markets and Human Computation: Advertising auctions, pricing, markets, and exchanges
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The optimization of bidding strategies for online advertising slot auctions presents a critical challenge across numerous digital marketplaces. A significant obstacle to the development, evaluation, and refinement of real-time autobidding algorithms is the scarcity of comprehensive datasets and standardized benchmarks. To address this deficiency, we present an auction benchmark encompassing the two most prevalent auction formats. We implement a series of robust baselines on a novel dataset, addressing the most salient Real-Time Bidding (RTB) problem domains: budget pacing uniformity and Cost Per Click (CPC) constraint optimization. This benchmark provides a user-friendly and intuitive framework for researchers and practitioners to develop and refine innovative autobidding algorithms, thereby facilitating advancements in the field of programmatic advertising. The implementation and additional resources can be accessed at the following repository https://github.com/avito-tech/bat-autobidding-benchmark, https://doi.org/10.5281/zenodo.14794182.

Problem

• Lack of comprehensive datasets for autobidding algorithm development
• Absence of standardized benchmarks for real-time bidding evaluation
• Challenges in budget pacing and CPC constraint optimization

Innovation

• Introduces auction benchmark for prevalent formats
• Implements robust baselines on novel dataset
• Provides user-friendly framework for autobidding algorithms