On the Existence of Unbiased Hypothesis Tests: An Algebraic Approach

2025-06-09

AI Summary

This paper investigates the existence of strictly unbiased hypothesis tests in discrete statistical models. It recasts the unbiasedness condition as a polynomial separation problem in real algebraic geometry, thereby establishing— for the first time—an algebraically decidable criterion and introducing the notion of the “unbiasedness threshold”: the minimal sample size guaranteeing existence of an unbiased test, shown to equal the degree of the minimal separating polynomial. Leveraging algebraic statistics and Gröbner basis theory, the paper derives necessary and sufficient algebraic conditions for the existence of unbiased tests and provides a constructive method to compute the unbiasedness threshold. It further reveals that the existence of uniformly most powerful unbiased (UMPU) tests exhibits dual sensitivity to both significance level and sample size. Explicit constructions of unbiased tests are supplied for canonical discrete models, including contingency tables, linear models, log-linear models, and mixed models.

Technology Category

• Reasoning under Uncertainty: Graphical Models
• Machine Learning: Probabilistic Circuits and Graphical Models
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Models for Web evolution

Abstract

In hypothesis testing problems the property of strict unbiasedness describes whether a test is able to discriminate, in the sense of a difference in power, between any distribution in the null hypothesis space and any distribution in the alternative hypothesis space. In this work we examine conditions under which unbiased tests exist for discrete statistical models. It is shown that the existence of an unbiased test can be reduced to an algebraic criterion; an unbiased test exists if and only if there exists a polynomial that separates the null and alternative hypothesis sets. This places a strong, semialgebraic restriction on the classes of null hypotheses that have unbiased tests. The minimum degree of a separating polynomial coincides with the minimum sample size that is needed for an unbiased test to exist, termed the unbiasedness threshold. It is demonstrated that Gr""obner basis techniques can be used to provide upper bounds for, and in many cases exactly find, the unbiasedness threshold. Existence questions for uniformly most powerful unbiased tests are also addressed, where it is shown that whether such a test exists can depend subtly on the specified level of the test and the sample size. Numerous examples, concerning tests in contingency tables, linear, log-linear, and mixture models are provided. All of the machinery developed in this work is constructive in the sense that when a test with a certain property is shown to exist it is possible to explicitly construct this test.

Problem

• Existence conditions for unbiased tests in discrete models
• Algebraic criterion for unbiased test existence via polynomial separation
• Determining unbiasedness threshold using Gröbner basis techniques

Innovation

• Algebraic criterion determines unbiased test existence
• Gröbner basis finds unbiasedness threshold
• Constructive method builds unbiased tests