Variational autoencoder for inference of nonlinear mixed effect models based on ordinary differential equations

2026-01-24

AI Summary

This work addresses the challenges of inference in nonlinear mixed-effects models (NLME) based on ordinary differential equations (ODEs) when applied to high-dimensional, sparse, or irregular longitudinal data, where complex, multimodal likelihood surfaces hinder the convergence and performance of traditional SAEM-MCMC algorithms. To overcome these limitations, we propose an amortized inference framework leveraging variational autoencoders (VAEs). By employing a shared encoder to maximize the evidence lower bound (ELBO), our approach efficiently estimates random effects without per-individual optimization or MCMC sampling. Parameter uncertainty is quantified via the observed information matrix, ensuring model identifiability while substantially improving inference efficiency and robustness. Experiments across three simulated scenarios—pharmacokinetics, vaccine immune response, and asthma-related TGF-β dynamics—as well as real-world antibody data demonstrate superior performance over conventional SAEM baselines.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We propose a variational autoencoder (VAE) approach for parameter estimation in nonlinear mixed-effects models based on ordinary differential equations (NLME-ODEs) using longitudinal data from multiple subjects. In moderate dimensions, likelihood-based inference via the stochastic approximation EM algorithm (SAEM) is widely used, but it relies on Markov Chain Monte-Carlo (MCMC) to approximate subject-specific posteriors. As model complexity increases or observations per subject are sparse and irregular, performance often deteriorates due to a complex, multimodal likelihood surface which may lead to MCMC convergence difficulties. We instead estimate parameters by maximizing the evidence lower bound (ELBO), a regularized surrogate for the marginal likelihood. A VAE with a shared encoder amortizes inference of subject-specific random effects by avoiding per-subject optimization and the use of MCMC. Beyond pointwise estimation, we quantify parameter uncertainty using observed-information-based variance estimator and verify that practical identifiability of the model parameters is not compromised by nuisance parameters introduced in the encoder. We evaluate the method in three simulation case studies (pharmacokinetics, humoral response to vaccination, and TGF-$\beta$ activation dynamics in asthmatic airways) and on a real-world antibody kinetics dataset, comparing against SAEM baselines.

Problem

• nonlinear mixed-effects models
• ordinary differential equations
• parameter estimation
• longitudinal data
• MCMC convergence

Innovation

• variational autoencoder
• nonlinear mixed-effects models
• ordinary differential equations
• evidence lower bound
• parameter uncertainty