Finite Sample Bounds for Learning with Score Matching

2026-05-13

AI Summary

This work addresses the lack of non-asymptotic sample complexity guarantees for learning high-dimensional continuous exponential family distributions, particularly in the challenging setting of unbounded support. By employing score matching to perform structure learning on polynomial-form exponential family models, the study establishes the first finite-sample error bounds for this class of models through a synthesis of high-dimensional probabilistic modeling and non-asymptotic statistical analysis. The results demonstrate that the required sample size scales polynomially with the ambient dimension, thereby providing the first rigorous sample complexity guarantee for learning high-dimensional exponential family distributions with unbounded support. This contribution fills a critical theoretical gap in the non-asymptotic understanding of continuous exponential families.

Abstract

Learning of continuous exponential family distributions with unbounded support remains an important area of research for both theory and applications in high-dimensional statistics. In recent years, score matching has become a widely used method for learning exponential families with continuous variables due to its computational ease when compared against maximum likelihood estimation. However, theoretical understanding of the statistical properties of score matching is still lacking. In this work, we provide a non-asymptotic sample complexity analysis for learning the structure of exponential families of polynomials with score matching. The derived sample bounds show a polynomial dependence on the model dimension. These bounds are the first of its kind, as all prior work has shown only asymptotic bounds on the sample complexity.

Problem

• score matching
• exponential families
• sample complexity
• non-asymptotic bounds
• high-dimensional statistics

Innovation

• score matching
• exponential families
• finite-sample bounds
• sample complexity
• non-asymptotic analysis