A complete characterization of testable hypotheses

2026-01-08

AI Summary

This study addresses the long-standing problem of characterizing the existence of nontrivial (strictly unbiased) hypothesis tests in the absence of a common dominating measure. By examining the closure of the convex hull of a set of probability measures within the space of bounded finitely additive measures, and leveraging topological separation properties under the total variation distance, the authors establish necessary and sufficient conditions for testability without requiring a reference control measure. This work completes the theoretical program initiated by Le Cam, providing the first complete characterization of testability in full generality. Illustrative examples further highlight the essential roles played by measure-theoretic and convex-analytic considerations in this foundational result.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Web Mining and Content Analysis: Web measurements

Abstract

We revisit a fundamental question in hypothesis testing: given two sets of probability measures $\mathcal{P}$ and $\mathcal{Q}$, when does a nontrivial (i.e.\ strictly unbiased) test for $\mathcal{P}$ against $\mathcal{Q}$ exist? Le~Cam showed that, when $\mathcal{P}$ and $\mathcal{Q}$ have a common dominating measure, a test that has power exceeding its level by more than $\varepsilon$ exists if and only if the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ are separated in total variation distance by more than $\varepsilon$. The requirement of a dominating measure is frequently violated in nonparametric statistics. In a passing remark, Le~Cam described an approach to address more general scenarios, but he stopped short of stating a formal theorem. This work completes Le~Cam's program, by presenting a matching necessary and sufficient condition for testability: for the aforementioned theorem to hold without assumptions, one must take the closures of the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ in the space of bounded finitely additive measures. We provide simple elucidating examples, and elaborate on various subtle measure theoretic and topological points regarding compactness and achievability.

Problem

• hypothesis testing
• testable hypotheses
• total variation distance
• convex hulls
• finitely additive measures

Innovation

• testability
• convex hull closure
• finitely additive measures
• total variation distance
• nonparametric hypothesis testing