Uncertainty-aware data assimilation through variational inference

2025-10-20

AI Summary

In data assimilation, uncertainty quantification remains challenging due to the coupling of dynamical models with process noise and sparse, noisy observations. To address this, we propose a variational inference–based uncertainty-aware data assimilation framework that models stochastic state evolution as a multivariate Gaussian distribution, enabling approximately perfectly calibrated uncertainty estimates and supporting longer assimilation windows. The method is end-to-end differentiable and seamlessly embeddable within machine learning architectures. Evaluated on the chaotic Lorenz-96 system, it achieves high state estimation accuracy while significantly improving uncertainty calibration and out-of-distribution generalization compared to conventional approaches. All code is publicly available to facilitate reproducibility and further research extensions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Intelligent Robots: State Estimation

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Data assimilation, consisting in the combination of a dynamical model with a set of noisy and incomplete observations in order to infer the state of a system over time, involves uncertainty in most settings. Building upon an existing deterministic machine learning approach, we propose a variational inference-based extension in which the predicted state follows a multivariate Gaussian distribution. Using the chaotic Lorenz-96 dynamics as a testing ground, we show that our new model enables to obtain nearly perfectly calibrated predictions, and can be integrated in a wider variational data assimilation pipeline in order to achieve greater benefit from increasing lengths of data assimilation windows. Our code is available at https://github.com/anthony-frion/Stochastic_CODA.

Problem

• Handling uncertainty in data assimilation with variational inference
• Improving prediction calibration using multivariate Gaussian distributions
• Extending data assimilation windows through stochastic modeling approach

Innovation

• Variational inference for uncertainty-aware data assimilation
• Multivariate Gaussian distribution for predicted states
• Improved calibration in chaotic Lorenz-96 dynamics