A Regret Perspective on Online Multiple Testing

2026-05-13

AI Summary

This work addresses the limitations of existing online multiple testing methods, which treat false discovery rate (FDR) control and statistical power in isolation and thus struggle with highly asymmetric costs of false positives and false negatives in automated settings. To overcome this, the authors propose a unified evaluation framework based on weighted regret and introduce Decoupled-OMT (DOMT), a meta-algorithm that ensures asymptotic FDR safety while mitigating threshold exhaustion during cold-start phases. The core contributions include establishing a regret conservation duality theory that reveals a linear lower bound on regret for deterministic FDR procedures, and incorporating a history-decoupled stochastic perturbation mechanism that achieves order-optimal regret reduction without incurring additional false negatives. Theoretical analysis shows that DOMT attains an Ω(√T) order-optimal regret bound under abrupt signal shifts, and experiments confirm its substantial reduction in empirical weighted regret and its ability to approach the non-stationary Pareto frontier.

Abstract

Online Multiple Testing (OMT), a fundamental pillar of sequential statistical inference, traditionally evaluates the False Discovery Rate (FDR) and statistical power in isolation, obscuring the highly asymmetric costs of false positives and false negatives in modern automated pipelines. To unify this evaluation, we introduce $\textit{Weighted Regret}$. Under this metric, we prove the $\textit{Duality of Regret Conservation}$: purely deterministic procedures ensuring strict FDR control inevitably incur an $Ω(T)$ linear regret penalty, as threshold depletion during signal-sparse cold starts forces massive false negatives. Tailored for exogenous testing streams, we propose Decoupled-OMT (DOMT) as a baseline-agnostic meta-wrapper. By incorporating a history-decoupled, strictly non-negative random perturbation, DOMT rescues purely deterministic baselines from severe threshold depletion. Crucially, it preserves exact asymptotic safety in stationary environments and rigorously bounds finite-sample error inflation during cold-starts. Guaranteeing zero additional false negatives, it yields an order-optimal $Ω(\sqrt{T})$ regret reduction in bursty environments, with a derived ``Cold-Start Tax'' characterizing the exact phase transition of algorithmic superiority. Experiments validate that DOMT consistently curtails empirical weighted regret, achieving an order-optimal sublinear mitigation of threshold depletion to navigate the non-stationary Pareto frontier.

Problem

• Online Multiple Testing
• False Discovery Rate
• Weighted Regret
• Threshold Depletion
• Cold Start

Innovation

• Weighted Regret
• Decoupled-OMT
• Threshold Depletion
• Cold-Start Tax
• Regret Conservation