Markovian Continuity of the MMSE

2025-04-20

AI Summary

This paper addresses a long-standing theoretical deficiency in minimum mean-square error (MMSE) estimation: its discontinuity under standard stochastic convergence. The root cause lies in “information overflow” within conventional convergent sequences—where some measurement conveys strictly more information about the quantity to be estimated than its limit does—violating fundamental modeling assumptions. To resolve this, the authors introduce *Markov convergence*, a novel notion of stochastic convergence requiring the measurement sequence and the target quantity to satisfy an asymptotic Markov condition. Under this framework, they establish, for the first time, the continuity of the MMSE estimator. Specifically, they prove continuity for linear estimators and upper semicontinuity for general nonlinear estimators. This work rectifies a critical limitation in classical estimation theory and provides a more realistic, robust foundation for convergence analysis in Bayesian estimation and signal processing.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Minimum mean square error (MMSE) estimation is widely used in signal processing and related fields. While it is known to be non-continuous with respect to all standard notions of stochastic convergence, it remains robust in practical applications. In this work, we review the known counterexamples to the continuity of the MMSE. We observe that, in these counterexamples, the discontinuity arises from an element in the converging measurement sequence providing more information about the estimand than the limit of the measurement sequence. We argue that this behavior is uncharacteristic of real-world applications and introduce a new stochastic convergence notion, termed Markovian convergence, to address this issue. We prove that the MMSE is, in fact, continuous under this new notion. We supplement this result with semi-continuity and continuity guarantees of the MMSE in other settings and prove the continuity of the MMSE under linear estimation.

Problem

• MMSE discontinuity under standard stochastic convergence notions
• Introducing Markovian convergence for MMSE continuity
• Proving MMSE continuity under linear estimation

Innovation

• Introduces Markovian convergence for MMSE continuity
• Proves MMSE continuity under new convergence notion
• Ensures MMSE robustness in practical applications