Small Ensemble-based Data Assimilation: A Machine Learning-Enhanced Data Assimilation Method with Limited Ensemble Size

2025-10-16

AI Summary

To address the significant degradation in filtering accuracy of ensemble-based methods under small ensemble sizes, this paper proposes an enhanced Ensemble Kalman Filter (EnKF) integrating a lightweight fully connected neural network (FCNN). The method first generates an initial analysis state using a small ensemble, then employs the FCNN to learn and predict the analysis error correction term, effectively compensating for accuracy loss with negligible additional computational cost. Its key innovation lies in the first incorporation of a compact FCNN into the EnKF framework, enabling synergistic optimization of error modeling and dynamic correction. Evaluated on the Lorenz-96 system and a nonlinear ocean wave field simulation, the proposed method achieves lower root-mean-square error (RMSE) than standard EnKF even when using only 20–50% of the conventional ensemble size. Moreover, it is readily compatible with other ensemble-based data assimilation algorithms and numerical models, offering high accuracy, low computational overhead, and strong generalizability.

Technology Category

• Machine Learning: Ensemble Methods
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Ensemble-based data assimilation (DA) methods have become increasingly popular due to their inherent ability to address nonlinear dynamic problems. However, these methods often face a trade-off between analysis accuracy and computational efficiency, as larger ensemble sizes required for higher accuracy also lead to greater computational cost. In this study, we propose a novel machine learning-based data assimilation approach that combines the traditional ensemble Kalman filter (EnKF) with a fully connected neural network (FCNN). Specifically, our method uses a relatively small ensemble size to generate preliminary yet suboptimal analysis states via EnKF. A FCNN is then employed to learn and predict correction terms for these states, thereby mitigating the performance degradation induced by the limited ensemble size. We evaluate the performance of our proposed EnKF-FCNN method through numerical experiments involving Lorenz systems and nonlinear ocean wave field simulations. The results consistently demonstrate that the new method achieves higher accuracy than traditional EnKF with the same ensemble size, while incurring negligible additional computational cost. Moreover, the EnKF-FCNN method is adaptable to diverse applications through coupling with different models and the use of alternative ensemble-based DA methods.

Problem

• Improves data assimilation accuracy with limited ensemble sizes
• Reduces computational costs while maintaining analysis precision
• Combines ensemble Kalman filter with neural network corrections

Innovation

• Combines ensemble Kalman filter with neural network
• Uses small ensemble for initial analysis states
• Learns correction terms to improve accuracy