Transformed Latent Variable Multi-Output Gaussian Processes

2026-05-06

AI Summary

This work addresses the scalability limitations of multi-output Gaussian processes (MOGPs) in high-dimensional output settings, where existing approaches often rely on restrictive assumptions such as low-rank structures or separable kernels, thereby compromising model expressiveness. The authors propose T-LVMOGP, a novel framework that introduces input- and output-specific latent variables and employs Lipschitz-regularized neural networks to map them into a shared embedding space, enabling a flexible deep multi-output kernel. Coupled with stochastic variational inference, this approach achieves efficient scalability without sacrificing the ability to capture complex inter-output dependencies. Empirical evaluations demonstrate that T-LVMOGP outperforms current baselines in both predictive accuracy and computational efficiency on challenging tasks, including climate modeling with over 10,000 outputs and zero-inflated spatial transcriptomics data.

Abstract

Multi-Output Gaussian Processes (MOGPs) provide a principled probabilistic framework for modelling correlated outputs but face scalability bottlenecks when applied to datasets with high-dimensional output spaces. To maintain tractability, existing methods typically resort to restrictive assumptions, such as employing low-rank or sum-of-separable kernels, which can limit expressiveness. We propose the Transformed Latent Variable MOGP (T-LVMOGP), a novel framework that scales MOGPs to a massive number of outputs while preserving the capacity to capture meaningful inter-output dependencies. T-LVMOGP constructs a flexible multi-output deep kernel by mapping inputs and output-specific latent variables into an embedding space using a Lipschitz-regularised neural network. Combined with stochastic variational inference, our model effectively scales to high-dimensional output settings. Across diverse benchmarks, including climate modelling with over 10,000 outputs and zero-inflated spatial transcriptomics data, T-LVMOGP outperforms baselines in both predictive accuracy and computational efficiency.

Problem

• Multi-Output Gaussian Processes
• Scalability
• High-dimensional Outputs
• Expressiveness
• Inter-output Dependencies

Innovation

• Multi-Output Gaussian Processes
• Latent Variable
• Deep Kernel
• Scalability
• Stochastic Variational Inference