Information Gap and Feasibility-Aware Inference in Binomial Logistic Mixtures

2026-06-14

AI Summary

This study addresses a fundamental limitation in binomial logistic mixture models: while mixture components are identifiable, the observed class labels are often not recoverable, leading standard inference methods to overestimate the number of components and produce poorly calibrated posterior probabilities. The authors demonstrate that this issue stems from an intrinsic information gap between component identifiability and label recoverability, arising from differences in local polynomial orders. To mitigate this, they propose two recoverability-aware inference approaches: a modified Bayesian Information Criterion (BIC) that incorporates recoverability constraints, and a maximum likelihood estimator regularized by posterior entropy. Numerical experiments show that both methods effectively prevent overfitting in component selection and substantially improve the reliability of label recovery and the calibration of predictive probabilities.

Abstract

This paper studies the information gap between mixture detection and label recovery in binomial logistic mixtures. Standard likelihood-based criteria such as the Bayesian information criterion (BIC) can detect the presence of two components, but this does not guarantee that the corresponding labels are recoverable. We show that this gap is intrinsic to binomial logistic mixtures with a fixed number of trials: observed-data evidence for mixture structure and per-observation information for label recovery have different local orders in the component separation, and only the former accumulates with the sample size. As a result, there exists a detectable-but-unrecoverable regime in which BIC selects two components while the posterior labels remain essentially uninformative. To address this issue, we propose two feasibility-aware inference procedures: a recoverability-aware BIC with a posterior-entropy penalty and an entropy-regularized estimator that mitigates the tendency of the maximum likelihood estimator to produce overly separated components and overly concentrated posterior responsibilities. Numerical experiments confirm the predicted gap and demonstrate that the proposed methods avoid misleading component selections and improve the calibration of posterior label probabilities.

Problem

• information gap
• label recovery
• binomial logistic mixtures
• mixture detection
• posterior entropy

Innovation

• information gap
• label recoverability
• binomial logistic mixtures
• feasibility-aware inference
• entropy regularization