Spatiotemporal System Forecasting with Irregular Time Steps via Masked Autoencoder

2026-03-26

AI Summary

This work addresses the challenge of predicting high-dimensional dynamical systems under irregular time steps, where data sparsity and the preservation of physical consistency pose significant difficulties. The authors propose a novel architecture that integrates a convolutional autoencoder with a masked autoencoder tailored for irregular time series, leveraging an attention mechanism to end-to-end reconstruct full physical fields without requiring data interpolation or explicit physical constraints. By uniquely combining masked autoencoding with spatiotemporal feature extraction, the method achieves state-of-the-art performance across multiple simulated and real-world ocean temperature datasets. It demonstrates substantial improvements over conventional convolutional and recurrent networks in terms of prediction accuracy, robustness to nonlinear dynamics, and computational efficiency.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Predicting high-dimensional dynamical systems with irregular time steps presents significant challenges for current data-driven algorithms. These irregularities arise from missing data, sparse observations, or adaptive computational techniques, reducing prediction accuracy. To address these limitations, we propose a novel method: a Physics-Spatiotemporal Masked Autoencoder. This method integrates convolutional autoencoders for spatial feature extraction with masked autoencoders optimised for irregular time series, leveraging attention mechanisms to reconstruct the entire physical sequence in a single prediction pass. The model avoids the need for data imputation while preserving physical integrity of the system. Here, 'physics' refers to high-dimensional fields generated by underlying dynamical systems, rather than the enforcement of explicit physical constraints or PDE residuals. We evaluate this approach on multiple simulated datasets and real-world ocean temperature data. The results demonstrate that our method achieves significant improvements in prediction accuracy, robustness to nonlinearities, and computational efficiency over traditional convolutional and recurrent network methods. The model shows potential for capturing complex spatiotemporal patterns without requiring domain-specific knowledge, with applications in climate modelling, fluid dynamics, ocean forecasting, environmental monitoring, and scientific computing.

Problem

• spatiotemporal forecasting
• irregular time steps
• high-dimensional dynamical systems
• missing data
• sparse observations

Innovation

• Masked Autoencoder
• Irregular Time Series
• Spatiotemporal Forecasting
• Physics-Informed Learning
• Attention Mechanism