Patch-GAN Transfer Learning with Reconstructive Models for Cloud Removal

2025-01-09

AI Summary

To address the problem of missing ground-object information in remote sensing imagery caused by cloud occlusion, this paper proposes a reconstruction-oriented transfer learning framework based on generative adversarial networks (GANs). Methodologically, we introduce masked autoencoders (MAEs) into cloud removal for the first time, leveraging their strong reconstruction priors to recover underlying scene structures beneath clouds; design a patch-based discriminator to enhance local texture fidelity assessment; and establish an end-to-end, reconstruction-driven transfer paradigm that jointly optimizes global structural consistency and local detail preservation. Evaluated on standard cloud removal benchmarks, our approach significantly outperforms existing GAN-based methods and achieves performance comparable to state-of-the-art non-GAN approaches. Quantitative and qualitative results demonstrate substantial improvements in both reconstruction accuracy of cloud-obscured ground objects and visual realism of restored imagery.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economic aspects and information design of Web, GenAI, and cloud computing
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Cloud removal plays a crucial role in enhancing remote sensing image analysis, yet accurately reconstructing cloud-obscured regions remains a significant challenge. Recent advancements in generative models have made the generation of realistic images increasingly accessible, offering new opportunities for this task. Given the conceptual alignment between image generation and cloud removal tasks, generative models present a promising approach for addressing cloud removal in remote sensing. In this work, we propose a deep transfer learning approach built on a generative adversarial network (GAN) framework to explore the potential of the novel masked autoencoder (MAE) image reconstruction model in cloud removal. Due to the complexity of remote sensing imagery, we further propose using a patch-wise discriminator to determine whether each patch of the image is real or not. The proposed reconstructive transfer learning approach demonstrates significant improvements in cloud removal performance compared to other GAN-based methods. Additionally, whilst direct comparisons with some of the state-of-the-art cloud removal techniques are limited due to unclear details regarding their train/test data splits, the proposed model achieves competitive results based on available benchmarks.

Problem

• Cloud Removal
• Remote Sensing Images
• Image Analysis Accuracy

Innovation

• Patch-GAN
• Masked Autoencoder (MAE)
• Cloud Removal