$Ω$: Operator-based Mixture Ensemble for Generative Assimilation

2026-06-18

AI Summary

This work addresses the challenges of data assimilation in high-dimensional nonlinear systems with partial observations, where traditional ensemble Kalman filters are limited by Gaussian assumptions, particle filters suffer from poor scalability, and existing generative approaches rely on inaccessible ground-truth posterior samples. The authors propose an unsupervised generative assimilation framework that leverages a conditional Gaussian surrogate model to analytically recover unobserved components, learns residual biases solely in an unsupervised manner, and reconstructs the full non-Gaussian posterior as a Gaussian mixture. By integrating denoising score matching with annealed Langevin dynamics to refine ensemble members, the method avoids dependence on true posterior samples, effectively mitigates the curse of dimensionality, and accurately captures multimodality, skewness, and heavy-tailed characteristics. It demonstrates superior performance over state-of-the-art assimilation techniques in turbulent models exhibiting intermittency and extreme events.

Abstract

Characterizing non-Gaussian posterior distributions in partially observed high-dimensional nonlinear systems remains a fundamental challenge in data assimilation. Ensemble Kalman filters rely on Gaussian approximations that can be inaccurate for strongly non-Gaussian posteriors, whereas particle filters suffer from severe scalability limitations. Recent score-based generative approaches improve posterior characterization but typically require supervised training with ground-truth posterior samples, which are unavailable in most practical applications. We introduce $Ω$ (Operator-based Mixture Ensemble for Generative Assimilation), a scalable framework that integrates conditional Gaussian surrogate modeling, unsupervised score learning, and generative sampling. The conditional Gaussian surrogate provides a nonlinear non-Gaussian baseline approximation while admitting closed-form conditional posterior distributions for the unresolved variables. First, $Ω$ exploits these closed-form conditional distributions to analytically recover the high-dimensional unobserved component, reducing computational cost and mitigating the curse of dimensionality. Second, $Ω$ learns only the residual discrepancy beyond an analytical baseline through denoising score matching using ensemble trajectories alone, eliminating the need for ground-truth posterior samples and substantially reducing the learning burden. Third, $Ω$ reconstructs the full non-Gaussian posterior distribution of both observed and unobserved variables via a Gaussian mixture representation, capturing multimodal, skewed, and heavy-tailed statistics. Finally, $Ω$ employs annealed Langevin sampling to iteratively refine ensemble members from the baseline toward the target posterior. $Ω$ is validated on several turbulent models with intermittency and extreme events, consistently improving posterior accuracy.

Problem

• non-Gaussian posterior
• data assimilation
• high-dimensional nonlinear systems
• ensemble methods
• score-based generative models

Innovation

• generative assimilation
• unsupervised score learning
• conditional Gaussian surrogate
• Gaussian mixture posterior
• annealed Langevin sampling