A Unified Study on Sequentiality in Universal Classification with Empirically Observed Statistics

2024-01-29

International Symposium on Information Theory

AI Summary

This paper investigates binary sequential classification under unknown distributions, optimizing the Type-II error exponent and expected stopping time subject to a distribution-dependent constraint λ(P₀,P₁) on the Type-I error exponent. We propose the first unified framework that imposes both constraints as universal requirements. Leveraging information-theoretic analysis, large deviations theory, and empirical process techniques, we derive the exact characterization of the optimal Type-II error exponent for both semi-sequential and fully sequential settings—establishing the first such precise expressions. Moreover, we prove that, when λ is continuous, sequential strategies can simultaneously achieve the theoretical upper bounds on both error exponents under certain conditions, thereby eliminating the inherent trade-off inherent in fixed-sample-size methods. The core contribution is the establishment of a universal sequential classification theory and the identification of necessary and sufficient conditions under which sequentiality attains optimal statistical efficiency.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

In hypothesis testing problems, taking samples sequentially and stopping opportunistically to make the inference greatly enhances the reliability. The design of the stopping and inference policy, however, critically relies on the knowledge of the underlying distribution of each hypothesis. When the knowledge of distributions, say, $P_{0}$ and $P_{1}$ in the binary-hypothesis case, is replaced by empirically observed statistics from the respective distributions, the gain of sequentiality is less understood when subject to universality constraints. In this work, the gap is mended by a unified study on sequentiality in the universal binary classification problem. We propose a unified framework where the universality constraints are set on the expected stopping time as well as the type-I error exponent. The type-I error exponent is required to achieve a pre-set distribution-dependent constraint $lambda(P_{0}, P_{1})$ for all $P_{0}, P_{1}$. The framework is employed to investigate a semi-sequential and a fully-sequential setup, so that fair comparison can be made with the fixed-length setup. The optimal type-II error exponents in different setups are characterized when the function $lambda$ satisfies mild continuity conditions. The benefit of sequentiality is shown by comparing the semi-sequential, the fully-sequential, and the fixed-length cases in representative examples of $lambda$. Conditions under which sequentiality eradicates the trade-off between error exponents are also derived.

Problem

• Sequential Methods
• Binary Classification
• Optimal Performance

Innovation

• Sequential Classification
• Optimal Error Probability
• Sequence Composite Hypothesis Testing