Testing hypotheses via orthogonalization

2026-06-28

AI Summary

Traditional hypothesis testing faces validity challenges in modern settings involving abstract null hypotheses, post hoc selection of inference targets, and weak distributional assumptions. This work proposes a novel approach that circumvents the need for pre-specified selection mechanisms or strong distributional assumptions. The method decomposes the data into two parts via symmetric noise injection and, under the null hypothesis, conditionally orthogonalizes one part with respect to the other; validity is then assessed by testing whether this orthogonality holds. Grounded in the theory of symmetric shift families, the procedure demonstrates broad applicability and flexibility across a variety of complex null hypotheses and post-selection scenarios, substantially expanding the scope of valid hypothesis testing.

Abstract

Classical hypothesis testing frameworks break down in contemporary settings in which null hypotheses are increasingly abstract, the same data are used to both generate and test hypotheses, and minimal assumptions about the underlying data are made. In this work, we propose a new framework for conducting valid hypothesis tests in broad contexts. We propose to add and subtract external noise generated from a symmetric shift-family to our data, $X$, to partition it into two pieces, $X^{(1)}$ and $X^{(2)}$. We provide a generic strategy for orthogonalizing $X^{(2)}$ against $X^{(1)}$ under the null hypothesis $H_0$, then show that testing whether the orthogonalization was successful provides a valid test of $H_0$ under mild assumptions. Remarkably, this framework extends naturally to the post-selection inference setting: we simply select a hypothesis on $X^{(1)}$, then perform orthogonalization under the selected null. As our approach neither requires pre-specification of the selection mechanism, nor is restricted to a small class of data-generating distributions, it dramatically expands the settings for which valid post-selection inference can be conducted. We showcase the flexibility of our proposal in several case studies involving challenging pre-specified null hypotheses and post-selection inference scenarios.

Problem

• hypothesis testing
• post-selection inference
• orthogonalization
• null hypothesis
• data splitting

Innovation

• orthogonalization
• post-selection inference
• hypothesis testing
• symmetric shift-family
• valid inference