Contrastive Heliophysical Image Pretraining for Solar Dynamics Observatory Records

2025-11-28

AI Summary

To address three key challenges in Solar Dynamics Observatory (SDO) multi-instrument data—cross-modal perception difficulty between AIA and HMI, weak inter-class separability, and high intra-class variability—this paper proposes SolarCHIP, a contrastive heliophysics image pretraining framework. Methodologically, SolarCHIP introduces a multi-granularity contrastive learning objective that jointly aligns: (i) global co-temporal multi-instrument representations, (ii) local patches at fixed spatial positions, and (iii) fine-grained intra-sample spatial structures—thereby learning modality-invariant and spatially consistent features. It integrates CNN/ViT-based autoencoders, contrastive learning, and ControlNet to enable both cross-modal translation and full-disk flare classification. Under low-resource settings, SolarCHIP significantly improves label efficiency, achieving state-of-the-art performance in both cross-modal translation and flare classification. The pretrained weights and source code are publicly released.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deep learning has revolutionized solar image analysis, yet most approaches train task-specific encoders from scratch or rely on natural-image pretraining that ignores the unique characteristics of Solar Dynamics Observatory (SDO) data. We introduce SolarCHIP, a family of contrastively pretrained visual backbones tailored to multi-instrument SDO observations. SolarCHIP addresses three key challenges in solar imaging: multimodal sensing across AIA and HMI instruments, weak inter-class separability due to slow temporal evolution, and strong intra-class variability with sparse activity signals. Our pretraining framework employs a multi-granularity contrastive objective that jointly aligns (1) global class tokens across co-temporal AIA-HMI pairs to enhance temporal discrimination, (2) local patch tokens at fixed spatial indices to enforce position-consistent, modality-invariant features, and (3) intra-sample patches across different spatial locations to preserve fine-grained spatial structure. We train both CNN- and Vision Transformer-based autoencoders and demonstrate their effectiveness on two downstream tasks: cross-modal translation between HMI and AIA passbands via ControlNet, and full-disk flare classification. Experimental results show that SolarCHIP achieves state-of-the-art performance across both tasks, with particularly strong gains in low-resource settings where labeled data is limited. Ablation studies confirm that each contrastive component contributes essential discriminative capacity at different granularities. By publicly releasing pretrained weights and training code, we provide the heliophysics community with a practical, plug-and-play feature extractor that reduces computational requirements, improves label efficiency, and establishes a reusable foundation for diverse solar imaging applications.

Problem

• Develops contrastive pretraining for multi-instrument solar image analysis
• Addresses weak class separability and high intra-class variability in solar data
• Enables cross-modal translation and flare classification with limited labeled data

Innovation

• Contrastive pretraining for multi-instrument solar data
• Multi-granularity contrastive objective aligning global and local features
• Autoencoders for cross-modal translation and flare classification