Spatio-temporal Fourier Transformer (StFT) for Long-term Dynamics Prediction

2025-03-14

AI Summary

Addressing the challenges of modeling multiscale coupling and insufficient uncertainty quantification in long-term dynamical forecasting of multiscale, multiphysics systems, this paper proposes the Spatiotemporal Fourier Transformer (StFT). StFT introduces a novel scale-aware Fourier-domain Transformer architecture, employing hierarchical Transformers to explicitly capture dynamic couplings across macroscopic and microscopic spatial scales, and integrates an uncertainty-driven generative residual correction mechanism to suppress error accumulation. It is the first method to jointly design structured multiscale modeling and uncertainty-aware correction. Evaluated on three canonical benchmarks—plasma dynamics, fluid dynamics, and atmospheric dynamics—StFT achieves 12.6%–28.3% higher long-term prediction accuracy than state-of-the-art machine learning methods and reduces uncertainty estimation error by 37.4%.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Simulating the long-term dynamics of multi-scale and multi-physics systems poses a significant challenge in understanding complex phenomena across science and engineering. The complexity arises from the intricate interactions between scales and the interplay of diverse physical processes. Neural operators have emerged as promising models for predicting such dynamics due to their flexibility and computational efficiency. However, they often fail to effectively capture multi-scale interactions or quantify the uncertainties inherent in the predictions. These limitations lead to rapid error accumulation, particularly in long-term forecasting of systems characterized by complex and coupled dynamics. To address these challenges, we propose a spatio-temporal Fourier transformer (StFT), in which each transformer block is designed to learn dynamics at a specific scale. By leveraging a structured hierarchy of StFT blocks, the model explicitly captures dynamics across both macro- and micro- spatial scales. Furthermore, a generative residual correction mechanism is integrated to estimate and mitigate predictive uncertainties, enhancing both the accuracy and reliability of long-term forecasts. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Problem

• Predict long-term dynamics in multi-scale, multi-physics systems.
• Capture multi-scale interactions and quantify prediction uncertainties.
• Improve accuracy and reliability of long-term forecasting.

Innovation

• Spatio-temporal Fourier Transformer for multi-scale dynamics
• Hierarchical StFT blocks capture macro-micro spatial scales
• Generative residual correction reduces predictive uncertainties