Efficient sequential Bayesian inference for state-space epidemic models using ensemble data assimilation

2025-12-05

AI Summary

Real-time epidemic response faces computational bottlenecks in inferring latent states and parameters from partially observed, noisy infectious disease data. Method: We propose eSMC², an efficient sequential Monte Carlo squared (SMC²) algorithm that replaces the internal particle filter in standard SMC² with the ensemble Kalman filter (EnKF), augmented by unbiased Gaussian density estimation and state-dependent observation variance—enhancing both scalability and accuracy, particularly for overdispersed epidemic data. The method integrates sequential Monte Carlo, EnKF, state-space modeling, and Bayesian updating to enable efficient incremental likelihood approximation. Results: Experiments on synthetic data and real 2022 U.S. monkeypox outbreak data demonstrate that eSMC² achieves posterior estimation accuracy comparable to conventional SMC², while reducing computational time substantially—enabling near-real-time reconstruction of transmission trajectories and identification of key epidemiological parameters.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Estimating latent epidemic states and model parameters from partially observed, noisy data remains a major challenge in infectious disease modeling. State-space formulations provide a coherent probabilistic framework for such inference, yet fully Bayesian estimation is often computationally prohibitive because evaluating the observed-data likelihood requires integration over all latent trajectories. The Sequential Monte Carlo squared (SMC$^2$) algorithm offers a principled approach for joint state and parameter inference, combining an outer SMC sampler over parameters with an inner particle filter that estimates the likelihood up to the current time point. Despite its theoretical appeal, this nested particle filter imposes substantial computational cost, limiting routine use in near-real-time outbreak response. We propose Ensemble SMC$^2$ (eSMC$^2$), a scalable variant that replaces the inner particle filter with an Ensemble Kalman Filter (EnKF) to approximate the incremental likelihood at each observation time. While this substitution introduces bias via a Gaussian approximation, we mitigate finite-sample effects using an unbiased Gaussian density estimator and adapt the EnKF for epidemic data through state-dependent observation variance. This makes our approach particularly suitable for overdispersed incidence data commonly encountered in infectious disease surveillance. Simulation experiments with known ground truth and an application to 2022 United States (U.S.) monkeypox incidence data demonstrate that eSMC$^2$ achieves substantial computational gains while producing posterior estimates comparable to SMC$^2$. The method accurately recovers latent epidemic trajectories and key epidemiological parameters, providing an efficient framework for sequential Bayesian inference from imperfect surveillance data.

Problem

• Estimates latent epidemic states and parameters from noisy, partial data.
• Reduces computational cost of Bayesian inference for real-time outbreak response.
• Handles overdispersed incidence data common in infectious disease surveillance.

Innovation

• Replaces inner particle filter with Ensemble Kalman Filter
• Uses unbiased Gaussian density estimator to reduce bias
• Adapts EnKF for epidemic data with state-dependent variance