Extreme bandits

2026-04-27

AI Summary

This work addresses the challenge of efficiently allocating limited resources to detect extreme-value sources characterized by the heaviest-tailed distributions—such as anomalous traffic spikes in network intrusion scenarios. It pioneers the extension of the multi-armed bandit framework to extreme value detection by introducing a sequential decision-making approach tailored for identifying extremal sources. Departing from conventional mean-maximization objectives, the study defines a novel "extreme regret" metric as the optimization target. Leveraging extreme value theory, the authors propose the ExtremeHunter algorithm, which dynamically allocates resources by integrating tail index estimation with an upper confidence bound strategy. Empirical evaluations on both synthetic and real-world datasets demonstrate that ExtremeHunter effectively approximates the optimal allocation policy and significantly outperforms existing baseline methods.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

In many areas of medicine, security, and life sciences, we want to allocate limited resources to different sources in order to detect extreme values. In this paper, we study an efficient way to allocate these resources sequentially under limited feedback. While sequential design of experiments is well studied in bandit theory, the most commonly optimized property is the regret with respect to the maximum mean reward. However, in other problems such as network intrusion detection, we are interested in detecting the most extreme value output by the sources. Therefore, in our work we study extreme regret which measures the efficiency of an algorithm compared to the oracle policy selecting the source with the heaviest tail. We propose the ExtremeHunter algorithm, provide its analysis, and evaluate it empirically on synthetic and real-world experiments.

Problem

• extreme bandits
• extreme regret
• sequential resource allocation
• heavy-tailed distributions
• extreme value detection

Innovation

• extreme bandits
• extreme regret
• heavy-tailed distributions
• sequential resource allocation
• ExtremeHunter