Long-Term Auto-Regressive Prediction using Lightweight AI Models: Adams-Bashforth Time Integration with Adaptive Multi-Step Rollout

2024-12-07

AI Summary

To address the challenge of long-term error accumulation in spatiotemporal autoregressive forecasting for scientific machine learning, this work proposes a novel framework integrating the Adams–Bashforth two-step explicit time integration scheme with adaptive multi-step rolling prediction. It innovatively incorporates classical numerical integration into autoregressive modeling, designs three dynamic weighted rolling training strategies, and introduces a lightweight graph neural network with only 1,177 parameters—enhanced by truncated grid generalization and noise-injection-based contrastive optimization. Evaluated on merely 50 training snapshots, the method achieves high-fidelity prediction over 350 steps (7:1 horizon) with a mean relative error of just 1.6%. Rolling prediction accuracy improves by 83% over standard noise injection, while maintaining strong robustness under challenging conditions such as sparse or truncated grids.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Multiagent Systems: Adversarial Agents
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This study addresses the critical challenge of error accumulation in spatio-temporal auto-regressive predictions within scientific machine learning models by introducing innovative temporal integration schemes and adaptive multi-step rollout strategies. We present a comprehensive analysis of time integration methods, highlighting the adaptation of the two-step Adams-Bashforth scheme to enhance long-term prediction robustness in auto-regressive models. Additionally, we improve temporal prediction accuracy through a multi-step rollout strategy that incorporates multiple future time steps during training, supported by three newly proposed approaches that dynamically adjust the importance of each future step. Despite using an extremely lightweight graph neural network with just 1,177 trainable parameters and training on only 50 snapshots, our framework accurately predicts 350 future time steps (a 7:1 prediction-to-training ratio) achieving an error of only 1.6% compared to the vanilla auto-regressive approach. Moreover, our framework demonstrates an 83% improvement in rollout performance over the standard noise injection method, a standard technique for enhancing long-term rollout performance. Its effectiveness is further validated in more challenging scenarios with truncated meshes, showcasing its adaptability and robustness in practical applications. This work introduces a versatile framework for robust long-term spatio-temporal auto-regressive predictions that shows potential for mitigating error accumulation across various model types and engineering disciplines.

Problem

• Addresses error accumulation in spatio-temporal auto-regressive predictions.
• Enhances long-term prediction robustness using Adams-Bashforth scheme.
• Improves temporal accuracy with adaptive multi-step rollout strategies.

Innovation

• Adaptive multi-step rollout strategy
• Two-step Adams-Bashforth time integration
• Lightweight graph neural network