Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator

2026-05-07

AI Summary

This work addresses the limitations of classical maximum likelihood estimation (MLE), whose asymptotic normality guarantees only distributional convergence without characterizing tail behavior, moment convergence, or information-theoretic rates. Under standard regularity conditions and a strengthened assumption on the score function, the paper establishes—for the first time—an explicit sub-Gaussian concentration inequality for the normalized MLE error and a central limit theorem in relative entropy. The analysis integrates exponential consistency bounds, high-order moment estimates, control of density derivatives, and information-theoretic tools. Notably, without requiring smoothness assumptions, it demonstrates that the normalized MLE exhibits sub-Gaussian tails, all moments converge to those of a Gaussian distribution, and, when either the Fisher information or the density derivatives are bounded, the estimator’s distribution converges to the Gaussian law in relative entropy.

Abstract

It is well known that, under standard regularity conditions, the maximum likelihood estimator (MLE) satisfies a central limit theorem and converges in distribution to a Gaussian random variable as the sample size grows. This paper strengthens this classical result by developing several stronger forms of asymptotic normality for the normalized MLE. With additional assumptions on the score, we first establish sub-Gaussian tail bounds and convergence of all moments for the normalized estimation error. We then prove an entropic central limit theorem for a smoothed version of the estimator, showing convergence in relative entropy to the limiting Gaussian law. When the Fisher information of the normalized estimate is bounded, or its density has bounded first derivative, we further show that the smoothing can be removed, yielding entropic normality of the MLE itself. The proofs develop auxiliary tools that may be of independent interest, including exponential consistency bounds, high-moment estimates, and entropy-control arguments for the estimator.

Problem

• maximum likelihood estimator
• asymptotic normality
• sub-Gaussian concentration
• entropic central limit theorem
• Fisher information

Innovation

• sub-Gaussian concentration
• entropic central limit theorem
• maximum likelihood estimator
• relative entropy
• asymptotic normality