Robust inference using density-powered Stein operators

2025-11-06

AI Summary

To address robust inference for unnormalized probability models, this paper introduces a density power-weighted γ-Stein operator framework—the first to incorporate γ-divergence into Stein methodology—enabling normalization-free and outlier-robust statistical inference. Methodologically, we construct a γ-Stein operator and derive two key tools: the γ-kernelized Stein discrepancy (for hypothesis testing) and γ-Stein variational gradient descent (for Bayesian inference), both inherently enjoying normalization-free computation and contamination robustness. Theoretically and empirically, our approach significantly outperforms conventional Stein methods and score matching baselines on contaminated Gaussian and quartic potential models, achieving superior trade-offs between robustness and statistical efficiency.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We introduce a density-power weighted variant for the Stein operator, called the $gamma$-Stein operator. This is a novel class of operators derived from the $gamma$-divergence, designed to build robust inference methods for unnormalized probability models. The operator's construction (weighting by the model density raised to a positive power $gamma$ inherently down-weights the influence of outliers, providing a principled mechanism for robustness. Applying this operator yields a robust generalization of score matching that retains the crucial property of being independent of the model's normalizing constant. We extend this framework to develop two key applications: the $gamma$-kernelized Stein discrepancy for robust goodness-of-fit testing, and $gamma$-Stein variational gradient descent for robust Bayesian posterior approximation. Empirical results on contaminated Gaussian and quartic potential models show our methods significantly outperform standard baselines in both robustness and statistical efficiency.

Problem

• Develops robust Stein operators for outlier-resistant statistical inference
• Creates robust score matching independent of normalizing constants
• Builds robust goodness-of-fit tests and Bayesian posterior approximation

Innovation

• Density-power weighted Stein operator for robustness
• Robust score matching independent of normalizing constant
• γ-Stein discrepancy and variational gradient descent applications