Multi-Output Robust and Conjugate Gaussian Processes

2025-10-30

AI Summary

Multi-output Gaussian processes (MOGPs) suffer from model misspecification and sensitivity to outliers; moreover, inter-output correlations exacerbate outlier propagation, severely degrading predictive robustness. To address this, we propose the Multi-Output Robust Conjugate Gaussian Process (MO-RCGP)—the first extension of the single-output RCGP framework to the multi-output setting. MO-RCGP jointly models cross-output dependencies while suppressing outlier propagation, preserving both conjugacy and Bayesian interpretability. We theoretically establish its statistical robustness against outliers. Its variational inference algorithm ensures computational efficiency and scalability. Empirical evaluation on financial time-series forecasting and cancer multi-omics modeling demonstrates that MO-RCGP significantly improves predictive stability and accuracy under outlier contamination, validating its effectiveness and practical utility in real-world, complex scenarios.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Multi-output Gaussian process (MOGP) regression allows modelling dependencies among multiple correlated response variables. Similarly to standard Gaussian processes, MOGPs are sensitive to model misspecification and outliers, which can distort predictions within individual outputs. This situation can be further exacerbated by multiple anomalous response variables whose errors propagate due to correlations between outputs. To handle this situation, we extend and generalise the robust and conjugate Gaussian process (RCGP) framework introduced by Altamirano et al. (2024). This results in the multi-output RCGP (MO-RCGP): a provably robust MOGP that is conjugate, and jointly captures correlations across outputs. We thoroughly evaluate our approach through applications in finance and cancer research.

Problem

• Modeling multi-output correlations with robustness to outliers
• Addressing error propagation across correlated anomalous response variables
• Extending robust conjugate Gaussian processes to multi-output scenarios

Innovation

• Multi-output robust Gaussian process for correlated variables
• Conjugate framework handling outliers and model misspecification
• Jointly captures correlations while ensuring robustness