ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations

2025-09-25

AI Summary

Chaos systems—such as weather and fluid dynamics—pose significant challenges for long-term forecasting due to extreme sensitivity to initial conditions and scarcity of observational data, resulting in poor zero-shot and few-shot prediction performance and weak generalization. To address this, we propose ChaosNexus, the first foundation model specifically designed for chaotic systems. Guided by a diversity-driven cross-system generalization principle, ChaosNexus introduces ScaleFormer—a novel multi-scale architecture—and integrates Mixture-of-Experts (MoE) layers, synergizing multi-scale representation learning, self-supervised pretraining, and few-shot fine-tuning. Evaluated on over 9,000 synthetic chaotic systems, it reduces long-term attractor statistical error by more than 40%. For global 5-day weather forecasting, it achieves zero-shot mean squared error below 1°C—substantially outperforming prior methods. This work pioneers the systematic adoption of the foundation model paradigm for chaos modeling, establishing a highly transferable, low-data-dependent universal forecasting framework for complex dynamical systems.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Accurately forecasting chaotic systems, prevalent in domains such as weather prediction and fluid dynamics, remains a significant scientific challenge. The inherent sensitivity of these systems to initial conditions, coupled with a scarcity of observational data, severely constrains traditional modeling approaches. Since these models are typically trained for a specific system, they lack the generalization capacity necessary for real-world applications, which demand robust zero-shot or few-shot forecasting on novel or data-limited scenarios. To overcome this generalization barrier, we propose ChaosNexus, a foundation model pre-trained on a diverse corpus of chaotic dynamics. ChaosNexus employs a novel multi-scale architecture named ScaleFormer augmented with Mixture-of-Experts layers, to capture both universal patterns and system-specific behaviors. The model demonstrates state-of-the-art zero-shot generalization across both synthetic and real-world benchmarks. On a large-scale testbed comprising over 9,000 synthetic chaotic systems, it improves the fidelity of long-term attractor statistics by more than 40% compared to the leading baseline. This robust performance extends to real-world applications with exceptional data efficiency. For instance, in 5-day global weather forecasting, ChaosNexus achieves a competitive zero-shot mean error below 1 degree, a result that further improves with few-shot fine-tuning. Moreover, experiments on the scaling behavior of ChaosNexus provide a guiding principle for scientific foundation models: cross-system generalization stems from the diversity of training systems, rather than sheer data volume.

Problem

• Forecasting chaotic systems across weather and fluid dynamics domains
• Overcoming generalization limitations in zero-shot chaotic system prediction
• Addressing data scarcity and sensitivity in chaotic dynamics modeling

Innovation

• Foundation model pre-trained on diverse chaotic dynamics
• Multi-scale architecture with Mixture-of-Experts layers
• Captures universal patterns and system-specific behaviors