Residual Deep Gaussian Processes on Manifolds

2024-10-31

arXiv.org

AI Summary

This work addresses the challenge of modeling complex non-stationary data—such as low-altitude wind velocity fields—on Riemannian manifolds. We propose Res-DGP, the first residual deep Gaussian process (DGP) framework explicitly designed for manifold-valued data. Res-DGP supports manifold-to-manifold latent mappings, enabling unified probabilistic modeling of manifold-valued outputs, scalar functions, and vector fields. Its key innovation is the integration of residual connections into manifold DGPs, enabling automatic complexity adaptation: it simplifies to a shallow GP under low-data-complexity regimes, thus balancing expressivity and robustness. The method rigorously combines Riemannian geometry, manifold differential geometry, and Bayesian uncertainty calibration. Experiments demonstrate that Res-DGP significantly improves prediction accuracy and uncertainty calibration on low-altitude wind field forecasting; accelerates late-stage convergence in manifold-constrained Bayesian optimization; and, for non-manifold data embeddable into proxy manifolds, expedites inference while preserving geometric fidelity.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We propose practical deep Gaussian process models on Riemannian manifolds, similar in spirit to residual neural networks. With manifold-to-manifold hidden layers and an arbitrary last layer, they can model manifold- and scalar-valued functions, as well as vector fields. We target data inherently supported on manifolds, which is too complex for shallow Gaussian processes thereon. For example, while the latter perform well on high-altitude wind data, they struggle with the more intricate, nonstationary patterns at low altitudes. Our models significantly improve performance in these settings, enhancing prediction quality and uncertainty calibration, and remain robust to overfitting, reverting to shallow models when additional complexity is unneeded. We further showcase our models on Bayesian optimisation problems on manifolds, using stylised examples motivated by robotics, and obtain substantial improvements in later stages of the optimisation process. Finally, we show our models to have potential for speeding up inference for non-manifold data, when, and if, it can be mapped to a proxy manifold well enough.

Problem

• Develop deep Gaussian process models for Riemannian manifolds.
• Improve prediction and uncertainty calibration for complex manifold data.
• Enhance Bayesian optimization on manifolds, particularly in robotics applications.

Innovation

• Deep Gaussian processes on Riemannian manifolds
• Manifold-to-manifold hidden layers
• Enhanced prediction quality and uncertainty calibration