ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics

2026-05-10

AI Summary

Current evaluations of spatiotemporal graph neural networks (STGNNs) are limited by reliance on a single real-world dataset and fixed data splits, hindering fair comparisons across diverse dynamical regimes. To address this, this work proposes the first tunable synthetic benchmark for chaotic lattice systems—termed the Chaotic Network Benchmark (CNB)—which leverages coupled standard maps to generate 96 distinct system configurations and 9,600 trajectories. By systematically controlling local chaos intensity, coupling strength, and system scale, CNB establishes a unified evaluation protocol. Experiments reveal that non-graph models outperform STGNNs in low-chaos scenarios, whereas STGNNs demonstrate superior robustness under high local and global chaos. These findings validate CNB as an effective, reproducible, and systematic platform for cross-architecture and cross-dynamical evaluation of spatiotemporal learning methods.

Abstract

Spatio-temporal graph neural networks (STGNNs) are widely used for short-term forecasting in dynamic physical systems such as traffic and weather. However, the prevailing evaluation practice uses real world benchmark data sets in a single domain with a single fixed holdout splits, making it difficult to compare architectures across different dynamical regimes. We introduce ChaosNetBench (CNB), a synthetic benchmark dataset and evaluation framework for studying STGNN performance under controlled multidimensional chaotic dynamics. CNB is built on a lattice of coupled standard maps with independently tunable local chaos ($K$), coupling strength ($\varepsilon$), and system size ($N$), providing known topology and known dynamics across 96 system instances and 9{,}600 trajectories. We introduce chaos indicators, evaluation metrics and a protocol to analyze and compare the capacity of STGNN architectures to deal with different levels of local and global chaos. We illustrate the usage of the framework by analyzing 13 architectures (5 STGNNs and 8 non-graph baselines). The results reveal a regime dependent transition in which non-graph baselines (TCN, N-BEATS, iTransformer) remain competitive when there is low local chaos, while STGNNs (e.g., Graph WaveNet, D2STGNN, STAEformer) are generally more resilient to higher levels of local and global chaos. CNB provides a practical, reusable testbed for systematically comparing and analyzing the capacity of STGNN architectures to handle different levels of local and global chaos.

Problem

• spatio-temporal graph neural networks
• benchmarking
• chaotic dynamics
• model evaluation
• dynamical systems

Innovation

• Spatio-temporal Graph Neural Networks
• Chaotic Dynamics
• Benchmarking Framework
• Synthetic Dataset
• Chaos Indicators