Surya: Foundation Model for Heliophysics

2025-08-18

AI Summary

Solar physics confronts challenges including task-specific modeling, scarcity of labeled data, and poor generalization. To address these, we introduce the first foundational model for solar physics: a spatiotemporal Transformer architecture (366M parameters) that jointly models full-resolution SDO/AIA and HMI multi-channel remote-sensing observations. Our method innovates with time-advancing prediction as the primary self-supervised pretraining objective and integrates spectral gating with hybrid long- and short-range attention mechanisms. We further employ autoregressive rollout pretraining and LoRA-based fine-tuning. Evaluated in zero-shot settings, the model successfully forecasts solar dynamical evolution and solar flare onset. It also achieves state-of-the-art performance on downstream tasks—including solar wind forecasting and active region segmentation—demonstrating its capacity to learn universal, physically grounded representations of solar evolutionary dynamics.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Heliophysics is central to understanding and forecasting space weather events and solar activity. Despite decades of high-resolution observations from the Solar Dynamics Observatory (SDO), most models remain task-specific and constrained by scarce labeled data, limiting their capacity to generalize across solar phenomena. We introduce Surya, a 366M parameter foundation model for heliophysics designed to learn general-purpose solar representations from multi-instrument SDO observations, including eight Atmospheric Imaging Assembly (AIA) channels and five Helioseismic and Magnetic Imager (HMI) products. Surya employs a spatiotemporal transformer architecture with spectral gating and long--short range attention, pretrained on high-resolution solar image forecasting tasks and further optimized through autoregressive rollout tuning. Zero-shot evaluations demonstrate its ability to forecast solar dynamics and flare events, while downstream fine-tuning with parameter-efficient Low-Rank Adaptation (LoRA) shows strong performance on solar wind forecasting, active region segmentation, solar flare forecasting, and EUV spectra. Surya is the first foundation model in heliophysics that uses time advancement as a pretext task on full-resolution SDO data. Its novel architecture and performance suggest that the model is able to learn the underlying physics behind solar evolution.

Problem

• Develops foundation model for solar activity forecasting
• Addresses limited generalization in task-specific heliophysics models
• Learns solar physics from multi-instrument SDO observations

Innovation

• Surya foundation model with spatiotemporal transformer architecture
• Pretrained on solar image forecasting tasks
• Uses Low-Rank Adaptation for efficient fine-tuning