Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions

2025-10-24

AI Summary

To address the challenge of surrogate modeling for high-dimensional piecewise-continuous functions, this paper proposes Deep Jump Gaussian Processes (DJGP). Methodologically, DJGP integrates local linear projection layers with jump Gaussian processes: region-adaptive local projection matrices enable structure-aware dimensionality reduction of the input space, while Gaussian process priors ensure spatially smooth evolution of projection parameters; a two-layer deep architecture, trained via variational inference and joint optimization, supports scalable learning and accurate uncertainty quantification. The key contribution is the first deep coupling of the geometric flexibility of local linear projections with the discontinuity-aware modeling capability of jump GPs. Experiments on synthetic and benchmark datasets demonstrate that DJGP significantly outperforms state-of-the-art surrogate models in both predictive accuracy and uncertainty calibration.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce Deep Jump Gaussian Processes (DJGP), a novel method for surrogate modeling of high-dimensional piecewise continuous functions. DJGP overcomes the limitations of conventional Jump Gaussian Processes in high-dimensional input spaces by adding a locally linear projection layer to Jump Gaussian Processes. This projection uses region-specific matrices to capture local subspace structures, naturally complementing the localized nature of JGP, a variant of local Gaussian Processes. To control model complexity, we place a Gaussian Process prior on the projection matrices, allowing them to evolve smoothly across the input space. The projected inputs are then modeled with a JGP to capture piecewise continuous relationships with the response. This yields a distinctive two-layer deep learning of GP/JGP. We further develop a scalable variational inference algorithm to jointly learn the projection matrices and JGP hyperparameters. Experiments on synthetic and benchmark datasets demonstrate that DJGP delivers superior predictive accuracy and more reliable uncertainty quantification compared to existing approaches.

Problem

• Surrogate modeling for high-dimensional piecewise continuous functions
• Overcoming limitations of conventional Jump Gaussian Processes in high dimensions
• Capturing local subspace structures with region-specific projection matrices

Innovation

• Deep Jump Gaussian Processes model high-dimensional piecewise functions
• Locally linear projection captures local subspace structures
• Variational inference jointly learns projections and hyperparameters