DDF: A Novel Dual-Domain Image Fusion Strategy for Remote Sensing Image Semantic Segmentation With Unsupervised Domain Adaptation

2024-03-05

IEEE Transactions on Geoscience and Remote Sensing

AI Summary

To address the dual challenges of scarce source-domain annotations and target-domain distribution shift in remote sensing image semantic segmentation, this paper proposes an unsupervised domain adaptation (UDA) framework based on dual-domain image fusion. The method integrates original images, style-transferred images, and intermediate-domain representations via a novel Dual-Domain Fusion (DDF) strategy. Additionally, it introduces a spatial-context-aware, region-specific pseudo-label reweighting mechanism to enhance the reliability of self-training. Evaluated on the ISPRS Vaihingen and Potsdam benchmarks, the approach achieves state-of-the-art performance. Ablation studies confirm the effectiveness and complementary nature of each component. This work delivers an efficient, robust, and interpretable solution for UDA in remote sensing semantic segmentation, advancing practical applicability under realistic annotation constraints.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The semantic segmentation of remote sensing (RS) images is a challenging and hot issue due to the large amount of unlabeled data and domain variation. Unsupervised domain adaptation (UDA) has proven to be advantageous in leveraging unlabeled information from the target domain. However, traditional approaches of independently fine-tuning UDA models in the source and target domains have a limited effect on the result. In this article, we propose a hybrid training strategy that boosts self-training methods with domain fusion images. First, we introduce a novel dual-domain image fusion (DDF) strategy to effectively utilize the original image, the style-transferred image, and the intermediate-domain information. Second, to further refine the precision of pseudolabels, we present a region-specific reweighting strategy that assigns different weights to pseudolabel regions based on their spatial context. Finally, we conduct a series of extensive benchmark experiments and ablation studies on the ISPRS Vaihingen and Potsdam datasets. These results show the efficiency of our approach and establish a practical basis for implementing semantic segmentation in remote sensors.

Problem

• Addresses semantic segmentation in remote sensing images
• Improves unsupervised domain adaptation for unlabeled data
• Enhances pseudo-label accuracy with region-specific weights

Innovation

• Hybrid training strategy for domain adaptation
• Dual-domain image fusion using multiple image types
• Pseudo-label region-specific weight strategy