Learning Representations of Satellite Images with Evaluations on Synoptic Weather Events

2025-08-08

AI Summary

This study addresses representation learning for satellite imagery to improve classification accuracy and reliability of critical weather events, particularly tropical cyclones. We propose a systematic evaluation framework comparing three representation learning approaches—Principal Component Analysis (PCA), Convolutional Autoencoders (CAEs), and pre-trained ResNet—across multi-scale and multi-resolution meteorological images to assess generalizability. Results show that CAEs achieve the highest Threat Score across all weather classification tasks, demonstrating superior modeling of meteorological semantic structure; higher input resolution consistently enhances performance; and excessively low latent dimensionality (<32) leads to a sharp increase in false positives. Our key contributions are: (1) the first adoption of Threat Score as the primary metric for evaluating representation effectiveness in meteorological contexts; and (2) empirical evidence that CAEs exhibit greater robustness than supervised pre-trained models under small-sample and weakly labeled meteorological conditions.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

This study applied representation learning algorithms to satellite images and evaluated the learned latent spaces with classifications of various weather events. The algorithms investigated include the classical linear transformation, i.e., principal component analysis (PCA), state-of-the-art deep learning method, i.e., convolutional autoencoder (CAE), and a residual network pre-trained with large image datasets (PT). The experiment results indicated that the latent space learned by CAE consistently showed higher threat scores for all classification tasks. The classifications with PCA yielded high hit rates but also high false-alarm rates. In addition, the PT performed exceptionally well at recognizing tropical cyclones but was inferior in other tasks. Further experiments suggested that representations learned from higher-resolution datasets are superior in all classification tasks for deep-learning algorithms, i.e., CAE and PT. We also found that smaller latent space sizes had minor impact on the classification task's hit rate. Still, a latent space dimension smaller than 128 caused a significantly higher false alarm rate. Though the CAE can learn latent spaces effectively and efficiently, the interpretation of the learned representation lacks direct connections to physical attributions. Therefore, developing a physics-informed version of CAE can be a promising outlook for the current work.

Problem

• Evaluating representation learning for satellite weather image classification
• Comparing PCA, CAE, and pre-trained networks for weather event detection
• Improving physics-informed deep learning for interpretable weather representations

Innovation

• Applied PCA, CAE, PT for satellite image analysis
• CAE latent space showed highest classification accuracy
• Physics-informed CAE proposed for future improvements