Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics

2025-02-03

AI Summary

Online advertising faces dual challenges: cold-start for new ads (inaccurate CTR estimation due to insufficient click data) and chronic CTR underestimation for mature ads (causing long-term revenue loss from early performance bias). Method: We propose a UCB-type multi-armed bandit algorithm tailored to the Position-Based Model (PBM) and integrated with the pay-per-click (PPC) auction mechanism. Contribution/Results: This is the first work to derive a theoretically provable upper bound on budgeted regret within the PBM framework. Our algorithm features a controllable exploration–exploitation trade-off, optimizing long-term platform revenue while satisfying short-term income constraints. Experiments on synthetic data and real-world ad logs demonstrate significant improvements: enhanced exposure efficiency and CTR estimation accuracy for new ads; 12.7% higher long-term revenue; and a 34% reduction in short-term income volatility.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Algorithm Configuration
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Online advertising platforms often face a common challenge: the cold start problem. Insufficient behavioral data (clicks) makes accurate click-through rate (CTR) forecasting of new ads challenging. CTR for"old"items can also be significantly underestimated due to their early performance influencing their long-term behavior on the platform. The cold start problem has far-reaching implications for businesses, including missed long-term revenue opportunities. To mitigate this issue, we developed a UCB-like algorithm under multi-armed bandit (MAB) setting for positional-based model (PBM), specifically tailored to auction pay-per-click systems. Our proposed algorithm successfully combines theory and practice: we obtain theoretical upper estimates of budget regret, and conduct a series of experiments on synthetic and real-world data that confirm the applicability of the method on the real platform. In addition to increasing the platform's long-term profitability, we also propose a mechanism for maintaining short-term profits through controlled exploration and exploitation of items.

Problem

• Address cold start in online advertising
• Improve CTR forecasting accuracy
• Balance exploration and exploitation in auctions

Innovation

• UCB-like algorithm application
• Multi-armed bandit framework
• Positional-based model optimization