Instance-Optimal Uniformity Testing and Tracking

2025-08-04

AI Summary

Traditional uniformity testing relies solely on a prespecified distance parameter, making it difficult to detect subtle or arbitrary deviations from uniformity in unknown underlying distributions. To address this, we introduce the new problem of *uniformity tracking*: dynamically detecting any form of non-uniformity with minimal samples under unknown distributions, while asymptotically matching the performance of an optimal offline algorithm that knows the true distribution structure post hoc. Method: We develop a structured theoretical framework based on Poissonized mixture models and design adaptive sampling strategies coupled with distribution profile estimation. Contribution/Results: Our approach achieves the first polylogarithmic competitive ratio—polylog(opt)—against the optimal offline algorithm. It overcomes limitations of gap-based models, significantly improves sample efficiency, and attains near-optimal detection power for *any* non-uniform distribution. This establishes a novel paradigm for lightweight, adaptive distribution testing.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Tracking, profiling, and countermeasures against them
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

In the uniformity testing task, an algorithm is provided with samples from an unknown probability distribution over a (known) finite domain, and must decide whether it is the uniform distribution, or, alternatively, if its total variation distance from uniform exceeds some input distance parameter. This question has received a significant amount of interest and its complexity is, by now, fully settled. Yet, we argue that it fails to capture many scenarios of interest, and that its very definition as a gap problem in terms of a prespecified distance may lead to suboptimal performance. To address these shortcomings, we introduce the problem of uniformity tracking, whereby an algorithm is required to detect deviations from uniformity (however they may manifest themselves) using as few samples as possible, and be competitive against an optimal algorithm knowing the distribution profile in hindsight. Our main contribution is a $operatorname{polylog}(operatorname{opt})$-competitive uniformity tracking algorithm. We obtain this result by leveraging new structural results on Poisson mixtures, which we believe to be of independent interest.

Problem

• Testing if unknown distribution is uniform or far
• Detecting deviations from uniformity efficiently
• Competitive algorithm for optimal uniformity tracking

Innovation

• Introduces uniformity tracking for dynamic deviation detection
• Leverages Poisson mixtures for structural analysis
• Achieves polylog(opt)-competitive algorithm performance