Bayesian Data Sketching for Varying Coefficient Regression Models

2025-05-30

AI Summary

Bayesian inference for varying-coefficient regression models with large-scale functional data is computationally prohibitive due to the high cost of Markov chain Monte Carlo (MCMC). To address this, we propose a data sketching method based on randomized linear transformations that compresses both the response vector and the predictor matrix into low-dimensional representations, while preserving the full Bayesian modeling framework. Standard MCMC or variational inference tools can then be directly applied to the sketched data. This work marks the first application of Bayesian data sketching to varying-coefficient models—requiring no new model specification, custom algorithm development, or specialized hardware. Theoretically and empirically, the method maintains near-identical statistical efficacy while achieving speedups of several orders of magnitude. Moreover, it seamlessly integrates with existing Bayesian inference ecosystems.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Data Compression

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Varying coefficient models are popular for estimating nonlinear regression functions in functional data models. Their Bayesian variants have received limited attention in large data applications, primarily due to prohibitively slow posterior computations using Markov chain Monte Carlo (MCMC) algorithms. We introduce Bayesian data sketching for varying coefficient models to obviate computational challenges presented by large sample sizes. To address the challenges of analyzing large data, we compress the functional response vector and predictor matrix by a random linear transformation to achieve dimension reduction and conduct inference on the compressed data. Our approach distinguishes itself from several existing methods for analyzing large functional data in that it requires neither the development of new models or algorithms, nor any specialized computational hardware while delivering fully model-based Bayesian inference. Well-established methods and algorithms for varying coefficient regression models can be applied to the compressed data.

Problem

• Speeding up Bayesian varying coefficient models for large data
• Compressing functional data to reduce computational complexity
• Enabling standard methods on compressed data without new algorithms

Innovation

• Bayesian data sketching for dimension reduction
• Random linear transformation compresses data
• Uses existing methods on compressed data