Nyström Kernel Stein Discrepancy

2024-06-12

arXiv.org

AI Summary

To address the $O(n^2)$ computational bottleneck of kernelized Stein discrepancy (KSD) under large-scale data—arising from its reliance on U- or V-statistics—this paper introduces, for the first time, the Nyström low-rank kernel approximation into KSD estimation, yielding a scalable and accelerated KSD estimator. The proposed method reduces time complexity to $O(mn + m^3)$, where $m ll n$, and establishes $sqrt{n}$-consistency under sub-Gaussian assumptions. Theoretical analysis is grounded in the Stein operator and reproducing kernel Hilbert space (RKHS) framework, balancing statistical efficiency with computational tractability. Extensive benchmark experiments demonstrate that the new estimator retains statistical power comparable to the original KSD while substantially enhancing practicality for large-scale goodness-of-fit testing. This work provides an efficient, theoretically sound tool for high-dimensional distribution fitting and hypothesis testing.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Kernel methods underpin many of the most successful approaches in data science and statistics, and they allow representing probability measures as elements of a reproducing kernel Hilbert space without loss of information. Recently, the kernel Stein discrepancy (KSD), which combines Stein's method with the flexibility of kernel techniques, gained considerable attention. Through the Stein operator, KSD allows the construction of powerful goodness-of-fit tests where it is sufficient to know the target distribution up to a multiplicative constant. However, the typical U- and V-statistic-based KSD estimators suffer from a quadratic runtime complexity, which hinders their application in large-scale settings. In this work, we propose a Nystr""om-based KSD acceleration -- with runtime $mathcal Oleft(mn+m^3 ight)$ for $n$ samples and $mll n$ Nystr""om points -- , show its $sqrt{n}$-consistency with a classical sub-Gaussian assumption, and demonstrate its applicability for goodness-of-fit testing on a suite of benchmarks.

Problem

• Accelerates Kernel Stein Discrepancy computation
• Reduces runtime complexity for large-scale data
• Ensures consistency in goodness-of-fit testing

Innovation

• Nyström-based KSD acceleration reduces runtime complexity.
• Achieves $sqrt{n}$-consistency with sub-Gaussian assumption.
• Applicable for large-scale goodness-of-fit testing.