A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression

2026-08-12

AI Summary

This work addresses the cubic computational complexity of multi-output Gaussian process regression and its difficulty in handling missing observations under heterogeneous inputs. The authors formulate the problem using a Forney-style factor graph, ordering inputs into a one-dimensional sequence via a nearest-neighbor chain and performing Gaussian message passing along this chain. This approach naturally accommodates missing data without requiring recomputation of the covariance matrix. By integrating Matérn latent processes, linear-Gaussian transition factors, and the linear model of coregionalization, the method achieves accuracy comparable to exact kernel methods in low-dimensional input settings. Evaluated on power time series forecasting tasks, the proposed approach matches baseline prediction accuracy while attaining linear scalability with respect to data size, thereby substantially improving computational efficiency.

Abstract

Multi-output Gaussian process regression scales cubically in the number of observations times outputs, and dense kernel-matrix methods need bespoke handling whenever different outputs are observed at different inputs. We express multi-output Gaussian process regression as a Forney-style factor graph in which a nearest-neighbor chain orders a fixed candidate set of $C$ inputs into a one-dimensional sequence. Along this chain, latent Matérn processes evolve through linear-Gaussian transition factors, while the linear model of coregionalization mixes $L$ latent processes into $D$ outputs through a deterministic mixing factor and per-output scalar observation factors. Posterior computation reduces to exact Gaussian message passing on the chain at cost $\mathcal{O}(C(DL^2 + L^3))$ after chain construction, and missing observations omit their local factor without any covariance-matrix restructuring. The formulation therefore scales in the number of data samples and in the rate of missing observations, while remaining best suited to candidate sets in low input dimension.We compare the factor-graph formulation against an exact kernel-matrix baseline, a sparse-variational inducing-point baseline, and a nearest-neighbor baseline on a synthetic input-dimension sweep and on electricity time series forecasting. At low input dimension the factor-graph posterior tracks the exact kernel-matrix posterior closely, and the gap grows gradually as input dimension increases while staying competitive with both approximate baselines. On the electricity time series our factor-graph formulation matches all three baselines in forecast accuracy while scaling linearly in the number of data points, where the exact kernel-matrix method becomes infeasible and the inducing-point baseline remains substantially slower.

Problem

• multi-output Gaussian process
• scalability
• missing observations
• factor graph
• computational complexity

Innovation

• factor graph
• multi-output Gaussian process
• nearest-neighbor chain
• linear model of coregionalization
• Gaussian message passing