It's all In the (Exponential) Family: An Equivalence between Maximum Likelihood Estimation and Control Variates for Sketching Algorithms

2026-01-29

AI Summary

This work addresses the limitations of maximum likelihood estimation (MLE) in sketching algorithms, which often suffers from numerical instability and computational inefficiency, and clarifies its previously unclear theoretical relationship with control variate estimation (CVE). Under exponential family distributions, the paper establishes—for the first time—the asymptotic equivalence between MLE and the optimal CVE by proving they share the same asymptotic variance. Building on this theoretical insight, the authors propose a novel MLE framework based on the expectation–maximization (EM) algorithm. This approach substantially improves numerical stability and computational efficiency. Empirical evaluations on bivariate normal distributions demonstrate its superiority over conventional root-finding methods, thereby validating the theoretical findings and highlighting its practical utility.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Maximum likelihood estimators (MLE) and control variate estimators (CVE) have been used in conjunction with known information across sketching algorithms and applications in machine learning. We prove that under certain conditions in an exponential family, an optimal CVE will achieve the same asymptotic variance as the MLE, giving an Expectation-Maximization (EM) algorithm for the MLE. Experiments show the EM algorithm is faster and numerically stable compared to other root finding algorithms for the MLE for the bivariate Normal distribution, and we expect this to hold across distributions satisfying these conditions. We show how the EM algorithm leads to reproducibility for algorithms using MLE / CVE, and demonstrate how the EM algorithm leads to finding the MLE when the CV weights are known.

Problem

• Maximum Likelihood Estimation
• Control Variates
• Exponential Family
• Sketching Algorithms
• Asymptotic Variance

Innovation

• Exponential Family
• Maximum Likelihood Estimation
• Control Variates
• Expectation-Maximization Algorithm
• Sketching Algorithms