Ensemble Controlled-Flow Filtering for Implicit Data Assimilation

2026-07-14

AI Summary

This work addresses the limitations of traditional ensemble filters in handling implicit, non-smooth, or many-to-one observation models, which typically rely on explicit likelihoods or observation derivatives. The authors propose an implicit data assimilation framework that defines the analysis distribution via energy tilting and introduces the Ensemble Control Flow filter (EnCF), which combines stochastic control flows with adjoint matching to learn observation-dependent control policies from terminal energy gradients. For simulator-defined observations, they further develop EnCF-LF to construct a conditional energy surrogate model. This approach uniquely integrates energy tilting with stochastic control flows, enabling filtering updates without requiring explicit likelihoods or derivatives, while theoretically ensuring that local errors do not accumulate. Experiments demonstrate significant performance gains over conventional Kalman-type filters in non-Gaussian, multimodal, and implicit observation settings.

Abstract

Data assimilation estimates the state of a dynamical system from model forecasts and incoming observations. Many observation mechanisms, however, are many-to-one, implicit, non-smooth, or accessible only through simulation, and need not provide the residual structures or likelihood guidance required by existing ensemble filters. We introduce implicit data assimilation, in which the analysis law is defined as an energy tilt of the forecast distribution. We then propose the Ensemble Controlled-flow Filter (EnCF), which realizes this update through a stochastic controlled flow and learns the observation-dependent control by adjoint matching from terminal energy gradients. For simulator-defined observations, EnCF-LF learns a surrogate conditional energy from samples and applies the same controlled-flow solver. We prove ideal exactness, derive a one-step error decomposition, and establish non-accumulation of local errors under filter stability. Numerical results show that Kalman-type filters remain preferable for smooth additive-Gaussian observations, while the proposed methods are better suited to non-Gaussian, many-to-one, multimodal, and implicit observation models.

Problem

• data assimilation
• implicit observations
• ensemble filtering
• non-Gaussian observations
• simulator-defined observations

Innovation

• implicit data assimilation
• ensemble controlled-flow filter
• stochastic controlled flow
• energy tilt
• adjoint matching