Partially Observable Gaussian Process Network and Doubly Stochastic Variational Inference

2025-02-19

AI Summary

In Gaussian Process Networks (GPNs), intermediate node observations are often indirect, noisy, and incomplete, hindering effective modeling and inference. To address this, we propose Partially Observable Gaussian Process Networks (POGPNs)—the first GPN framework explicitly incorporating partial observability. We introduce an *observation lens* mechanism to characterize non-ideal observation processes and develop a doubly stochastic variational inference framework enabling end-to-end training and joint optimization of latent functions across nodes. POGPN requires no ground-truth labels at intermediate nodes and naturally integrates sparse, heterogeneous, and noisy intermediate observations. Experiments on multiple benchmark tasks demonstrate significant improvements in overall predictive accuracy and generalization performance. Our approach establishes a new paradigm for modeling high-dimensional nonlinear systems under partial observability.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Probabilistic Circuits and Graphical Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

To reduce the curse of dimensionality for Gaussian processes (GP), they can be decomposed into a Gaussian Process Network (GPN) of coupled subprocesses with lower dimensionality. In some cases, intermediate observations are available within the GPN. However, intermediate observations are often indirect, noisy, and incomplete in most real-world systems. This work introduces the Partially Observable Gaussian Process Network (POGPN) to model real-world process networks. We model a joint distribution of latent functions of subprocesses and make inferences using observations from all subprocesses. POGPN incorporates observation lenses (observation likelihoods) into the well-established inference method of deep Gaussian processes. We also introduce two training methods for POPGN to make inferences on the whole network using node observations. The application to benchmark problems demonstrates how incorporating partial observations during training and inference can improve the predictive performance of the overall network, offering a promising outlook for its practical application.

Problem

• Reduces dimensionality curse for Gaussian processes
• Models real-world process with partial observations
• Improves predictive performance using partial observations

Innovation

• Partially Observable Gaussian Process Network
• Doubly Stochastic Variational Inference
• Incorporates observation lenses into inference