BITS for GAPS: Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates

2025-11-20

AI Summary

Modeling hybrid physical systems—where part of the dynamics follows known physical laws while the remainder is governed by unknown, data-driven latent dynamics—remains challenging. Method: We propose a Bayesian information-theoretic sampling framework: (i) a hierarchical Gaussian process (GP) prior encodes domain-specific physical structure; (ii) we derive closed-form expressions and computable lower bounds for the predictive posterior differential entropy, enabling an entropy-guided active sampling strategy; and (iii) we integrate a finite uniformly weighted GP mixture approximation, incorporating extended Raoult’s law constraints for thermodynamically consistent vapor–liquid equilibrium modeling. Contribution/Results: This work is the first to explicitly leverage differential entropy for sample selection in hybrid modeling. It significantly improves prediction accuracy—particularly in non-ideal phase-equilibrium regions—and accelerates surrogate model convergence, while rigorously enforcing physical consistency. Experimental results demonstrate the framework’s efficiency and reliability in real-world engineering applications.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We introduce the Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates (BITS for GAPS) framework to emulate latent components in hybrid physical systems. BITS for GAPS supports serial hybrid modeling, where known physics governs part of the system and residual dynamics are represented as a latent function inferred from data. A Gaussian process prior is placed over the latent function, with hierarchical priors on its hyperparameters to encode physically meaningful structure in the predictive posterior. To guide data acquisition, we derive entropy-based acquisition functions that quantify expected information gain from candidate input locations, identifying samples most informative for training the surrogate. Specifically, we obtain a closed-form expression for the differential entropy of the predictive posterior and establish a tractable lower bound for efficient evaluation. These derivations approximate the predictive posterior as a finite, uniformly weighted mixture of Gaussian processes. We demonstrate the framework's utility by modeling activity coefficients in vapor-liquid equilibrium systems, embedding the surrogate into extended Raoult's law for distillation design. Numerical results show that entropy-guided sampling improves sample efficiency by targeting regions of high uncertainty and potential information gain. This accelerates surrogate convergence, enhances predictive accuracy in non-ideal regimes, and preserves physical consistency. Overall, BITS for GAPS provides an efficient, interpretable, and uncertainty-aware framework for hybrid modeling of complex physical systems.

Problem

• Emulating latent components in hybrid physical systems using Gaussian processes
• Deriving entropy-based acquisition functions to guide data sampling
• Improving sample efficiency and predictive accuracy in non-ideal physical regimes

Innovation

• Bayesian information-theoretic sampling for hierarchical Gaussian processes
• Entropy-based acquisition functions guide data sampling
• Hybrid modeling combines physics with data-driven latent functions