Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

2026-07-06

AI Summary

Existing Bayesian generalized linear mixed models (GLMMs) struggle to accommodate high-dimensional multimodal data such as images and text. This work proposes a conditional Bayesian framework that jointly trains modality-specific neural encoders with a GLMM, employing variance-corrected stochastic gradient Markov chain Monte Carlo (SG-MCMC) for posterior inference. The approach uniquely integrates supervised representation learning with principled uncertainty quantification within the GLMM paradigm, preserving interpretability of fixed effects, random effects, and modality-specific slopes while supporting multimodal inputs. Experiments demonstrate that the method accurately recovers full-data MCMC posterior distributions in simulations and achieves strong predictive performance, well-calibrated uncertainty estimates, and meaningful quantification of modality importance in real-world applications to glaucoma progression and adolescent mental health prediction.

Abstract

Scalable Bayesian inference for generalized linear mixed models (GLMMs) provides uncertainty-aware analysis of correlated longitudinal data, but existing scalable approaches largely assume low-dimensional tabular predictors and do not directly accommodate high-dimensional modalities such as images and text. We address this limitation by learning one or more modality-specific neural encoders jointly with a GLMM objective, then performing variance-corrected stochasticgradient MCMC for the GLMM parameters conditional on the learned representation. This conditional-Bayes design combines supervised representation learning with posterior uncertainty quantification for population-level effects, subjectspecific heterogeneity, and modality-level random slopes. The resulting model preserves interpretable fixed and random effects for structured covariates and learned modalities while scaling gracefully to large longitudinal datasets. In simulation studies, our method recovers posterior means and variance estimates from full-data MCMC benchmarks after covariance correction. We further evaluate uncertainty through parameter-level interval coverage in simulations and predictive calibration on held-out data. Applications to glaucoma progression and adolescent mental health demonstrate that the framework allows nuanced assessment of the relative importance of each modality on both individual and population levels without sacrificing predictive performance.

Problem

• Bayesian inference
• generalized linear mixed models
• multimodal data
• high-dimensional modalities
• longitudinal data

Innovation

• neural encoders
• Bayesian GLMM
• multimodal data
• stochastic gradient MCMC
• uncertainty quantification