gp2Scale: A Class of Compactly-Supported Non-Stationary Kernels and Distributed Computing for Exact Gaussian Processes on 10 Million Data Points

2025-12-05

AI Summary

Existing Gaussian process (GP) methods struggle to simultaneously achieve computational efficiency, predictive accuracy, and model customizability on large-scale datasets—primarily due to reliance on approximations that compromise uncertainty quantification, restrict kernel and noise model design, and hinder effective modeling of expressive nonstationary patterns. This paper introduces gp2Scale: a framework that automatically constructs sparse covariance structures via flexible, compactly supported nonstationary kernels, enabling exact GP inference on datasets with millions of observations without any approximation. By integrating distributed computing with advanced sparse linear algebra techniques, gp2Scale efficiently solves linear systems and computes log-determinants. The method supports arbitrary kernel functions, noise models, and input space types—including irregular or high-dimensional domains. Empirical evaluation across multiple real-world benchmarks demonstrates that gp2Scale significantly outperforms or matches state-of-the-art approximate GP methods, thereby overcoming a fundamental bottleneck in scalable exact GP modeling.

Technology Category

• Machine Learning: Scalability of ML Systems
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Despite a large corpus of recent work on scaling up Gaussian processes, a stubborn trade-off between computational speed, prediction and uncertainty quantification accuracy, and customizability persists. This is because the vast majority of existing methodologies exploit various levels of approximations that lower accuracy and limit the flexibility of kernel and noise-model designs -- an unacceptable drawback at a time when expressive non-stationary kernels are on the rise in many fields. Here, we propose a methodology we term emph{gp2Scale} that scales exact Gaussian processes to more than 10 million data points without relying on inducing points, kernel interpolation, or neighborhood-based approximations, and instead leveraging the existing capabilities of a GP: its kernel design. Highly flexible, compactly supported, and non-stationary kernels lead to the identification of naturally occurring sparse structure in the covariance matrix, which is then exploited for the calculations of the linear system solution and the log-determinant for training. We demonstrate our method's functionality on several real-world datasets and compare it with state-of-the-art approximation algorithms. Although we show superior approximation performance in many cases, the method's real power lies in its agnosticism toward arbitrary GP customizations -- core kernel design, noise, and mean functions -- and the type of input space, making it optimally suited for modern Gaussian process applications.

Problem

• Scaling exact Gaussian processes to over 10 million data points without approximations.
• Enabling flexible non-stationary kernel designs while maintaining computational efficiency.
• Overcoming trade-offs between speed, accuracy, and customizability in GP methods.

Innovation

• Compactly-supported non-stationary kernels enable sparse covariance matrices
• Exploits kernel design for exact Gaussian processes without approximations
• Scales to over 10 million data points using distributed computing