Scalable Learning of Multivariate Distributions via Coresets

2026-03-20

AI Summary

This work proposes the first coreset construction for multivariate conditional transformation models (MCTMs) to address the computational inefficiency of existing nonparametric and semiparametric regression and density estimation methods on large-scale data. By integrating importance sampling with a convex hull–based geometric approximation, the approach effectively tackles the intractable log-normalization constant and guarantees, with high probability, that the log-likelihood error remains within a $(1\pm\varepsilon)$ multiplicative factor of the true value. The resulting coreset significantly enhances training efficiency and scalability while preserving statistical accuracy and improving the model’s capacity to capture complex nonlinear distributions. This advancement establishes a new paradigm for scalable and accurate distribution modeling in high-dimensional settings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Efficient and scalable non-parametric or semi-parametric regression analysis and density estimation are of crucial importance to the fields of statistics and machine learning. However, available methods are limited in their ability to handle large-scale data. We address this issue by developing a novel coreset construction for multivariate conditional transformation models (MCTMs) to enhance their scalability and training efficiency. To the best of our knowledge, these are the first coresets for semi-parametric distributional models. Our approach yields substantial data reduction via importance sampling. It ensures with high probability that the log-likelihood remains within multiplicative error bounds of $(1\pm\varepsilon)$ and thereby maintains statistical model accuracy. Compared to conventional full-parametric models, where coresets have been incorporated before, our semi-parametric approach exhibits enhanced adaptability, particularly in scenarios where complex distributions and non-linear relationships are present, but not fully understood. To address numerical problems associated with normalizing logarithmic terms, we follow a geometric approximation based on the convex hull of input data. This ensures feasible, stable, and accurate inference in scenarios involving large amounts of data. Numerical experiments demonstrate substantially improved computational efficiency when handling large and complex datasets, thus laying the foundation for a broad range of applications within the statistics and machine learning communities.

Problem

• scalability
• multivariate distributions
• large-scale data
• semi-parametric models
• density estimation

Innovation

• coresets
• multivariate conditional transformation models
• semi-parametric distributional models
• importance sampling
• scalable density estimation