Minimum L2 and robust Kullback-Leibler estimation

2026-02-20

AI Summary

This study addresses the robustness of parameter estimation under outliers or model misspecification by proposing two novel approaches: minimum weighted L² estimation and robust Kullback–Leibler estimation. These methods achieve robustness through weighted least squares and a robustified empirical Kullback–Leibler divergence, respectively, thereby extending maximum likelihood estimation in a robust manner and establishing theoretical connections to local likelihood principles. The authors derive analytical expressions for the influence function and asymptotic variance, integrating tools from robust statistical inference, influence function analysis, asymptotic efficiency calculations, and semiparametric density estimation. Under normal models, the proposed estimators demonstrate high asymptotic efficiency when the model is correctly specified, while exhibiting superior robustness and statistical performance in the presence of contaminated data.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Adversarial Learning & Robustness
• Reasoning under Uncertainty: Graphical Models

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

This paper introduces two new robust methods for estimation of parameters in a given parametric family. The first method is that of `minimum weighted L2', effectively minimising an estimate of the integrated (and possibly weighted) squared error. The second is `robust Kullback-Leibler', consisting of minimising a robust version of the empirical Kullback-Leibler distance, and can be viewed as a general robust modification of the maximum likelihood procedure. This second method is also related to recent local likelihood ideas for semiparametric density estimation. The methods are described, influence functions are found, as are formulae for asymptotic variances. In particular large-sample efficiencies are computed under the home turf conditions of the underlying parametric model. The methods and formulae are illustrated for the normal model.

Problem

• robust estimation
• parameter estimation
• Kullback-Leibler divergence
• L2 estimation
• maximum likelihood

Innovation

• robust estimation
• minimum weighted L2
• robust Kullback-Leibler
• influence function
• asymptotic efficiency