Enhanced accuracy through ensembling of randomly initialized auto-regressive models for time-dependent PDEs

2025-07-04

AI Summary

In autoregressive long-term forecasting of PDE-driven spatiotemporal fields, error accumulation severely degrades prediction accuracy. To address this, we propose a deep ensemble learning framework based on stochastic initialization. Under an autoregressive neural architecture, multiple independently initialized models are trained in parallel, and their predictions are aggregated during inference—thereby preserving temporal dependency modeling while enhancing robustness via ensemble averaging. Crucially, no additional time-step inputs are required to suppress error propagation. Evaluated on three canonical PDE systems—heterogeneous microstructural stress evolution, the Gray–Scott reaction-diffusion system, and the shallow water equations—our method reduces long-term prediction error by 32%–57% on average compared to single-model baselines, significantly improving trajectory fidelity. Moreover, inference is accelerated by 2–3 orders of magnitude relative to traditional numerical solvers. The implementation is publicly available.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Systems governed by partial differential equations (PDEs) require computationally intensive numerical solvers to predict spatiotemporal field evolution. While machine learning (ML) surrogates offer faster solutions, autoregressive inference with ML models suffer from error accumulation over successive predictions, limiting their long-term accuracy. We propose a deep ensemble framework to address this challenge, where multiple ML surrogate models with random weight initializations are trained in parallel and aggregated during inference. This approach leverages the diversity of model predictions to mitigate error propagation while retaining the autoregressive strategies ability to capture the system's time dependent relations. We validate the framework on three PDE-driven dynamical systems - stress evolution in heterogeneous microstructures, Gray-Scott reaction-diffusion, and planetary-scale shallow water system - demonstrating consistent reduction in error accumulation over time compared to individual models. Critically, the method requires only a few time steps as input, enabling full trajectory predictions with inference times significantly faster than numerical solvers. Our results highlight the robustness of ensemble methods in diverse physical systems and their potential as efficient and accurate alternatives to traditional solvers. The codes for this work are available on GitHub (https://github.com/Graham-Brady-Research-Group/AutoregressiveEnsemble_SpatioTemporal_Evolution).

Problem

• Mitigates error accumulation in autoregressive ML models for PDEs
• Improves long-term accuracy of spatiotemporal field predictions
• Reduces computational cost compared to traditional numerical solvers

Innovation

• Ensemble of randomly initialized autoregressive models
• Diverse predictions mitigate error propagation
• Few time steps enable full trajectory predictions