Testing Imprecise Hypotheses

2025-10-23

AI Summary

Scientific theories often only partially specify underlying distributions; thus, strict goodness-of-fit testing—requiring data to be exactly drawn from a candidate distribution—is unrealistic. This work addresses the fundamental trade-off between model misspecification tolerance (i.e., the radius of a neighborhood around the null distribution, measured under smooth or non-smooth norms) and test power. Method: We formulate a tolerant goodness-of-fit testing framework, where the null hypothesis asserts that the true distribution lies within a prescribed distance of the candidate model. Contribution/Results: We establish the exact information-theoretic rates of this trade-off for three canonical nonparametric settings: Gaussian sequence models, smooth regression, and density estimation. We prove that classical chi-square tests are suboptimal in this framework. Leveraging these characterizations, we construct simple, computationally efficient tests that achieve the fundamental information-theoretic limits, markedly improving robustness to plausible model deviations while preserving high detection power.

Technology Category

• Machine Learning: Information Theory
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements

Abstract

Many scientific applications involve testing theories that are only partially specified. This task often amounts to testing the goodness-of-fit of a candidate distribution while allowing for reasonable deviations from it. The tolerant testing framework provides a systematic way of constructing such tests. Rather than testing the simple null hypothesis that data was drawn from a candidate distribution, a tolerant test assesses whether the data is consistent with any distribution that lies within a given neighborhood of the candidate. As this neighborhood grows, the tolerance to misspecification increases, while the power of the test decreases. In this work, we characterize the information-theoretic trade-off between the size of the neighborhood and the power of the test, in several canonical models. On the one hand, we characterize the optimal trade-off for tolerant testing in the Gaussian sequence model, under deviations measured in both smooth and non-smooth norms. On the other hand, we study nonparametric analogues of this problem in smooth regression and density models. Along the way, we establish the sub-optimality of the classical chi-squared statistic for tolerant testing, and study simple alternative hypothesis tests.

Problem

• Testing hypotheses with partially specified theoretical models
• Characterizing trade-off between tolerance and test power
• Establishing sub-optimality of classical chi-squared statistic

Innovation

• Tolerant testing framework for imprecise hypotheses
• Characterizes trade-off between neighborhood size and power
• Establishes sub-optimality of classical chi-squared statistic