Multi-modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimation

2025-09-25

AI Summary

To address the low modeling efficiency and difficulty in uncertainty quantification for high-cost objective functions under multimodal data, this paper proposes two multimodal Bayesian neural network (BNN) surrogate models. The core methodological innovation is a conjugate final-layer design, enabling closed-form parameter updates and efficient variational inference while ensuring robustness to partial modality missing. By integrating multimodal feature encoding, stochastic variational inference, and conjugate distribution assumptions, the approach significantly improves prediction accuracy and uncertainty calibration on both scalar and time-series tasks—outperforming unimodal BNN baselines. The framework is modular and seamlessly embeddable into outer-loop applications such as optimization and inverse problem solving, thereby enhancing modeling efficiency, generalizability, and decision reliability for complex systems.

Technology Category

• Machine Learning: Multimodal Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

As data collection and simulation capabilities advance, multi-modal learning, the task of learning from multiple modalities and sources of data, is becoming an increasingly important area of research. Surrogate models that learn from data of multiple auxiliary modalities to support the modeling of a highly expensive quantity of interest have the potential to aid outer loop applications such as optimization, inverse problems, or sensitivity analyses when multi-modal data are available. We develop two multi-modal Bayesian neural network surrogate models and leverage conditionally conjugate distributions in the last layer to estimate model parameters using stochastic variational inference (SVI). We provide a method to perform this conjugate SVI estimation in the presence of partially missing observations. We demonstrate improved prediction accuracy and uncertainty quantification compared to uni-modal surrogate models for both scalar and time series data.

Problem

• Developing multi-modal Bayesian neural network surrogates with conjugate estimation
• Handling partially missing observations in multi-modal data integration
• Improving prediction accuracy and uncertainty quantification over uni-modal models

Innovation

• Multi-modal Bayesian neural network surrogate models
• Conjugate last-layer estimation with SVI
• Handles partially missing observations