Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation

2025-04-08

AI Summary

Severe artifacts in remote sensing (RS) imagery degrade the generalization of existing parameter-efficient fine-tuning (PEFT) methods. To address this, we propose the first frequency-domain PEFT framework tailored for RS scenarios. Our method innovatively integrates discrete Fourier transform (DFT) with mixture-of-adapters (MoA), establishing a hybrid frequency-adaptive mechanism that decouples artifact and semantic features in the frequency domain and enables dynamic weighted fusion. The framework requires no modification to the backbone architecture, significantly enhancing the robustness and transferability of foundation models on geospatial tasks. Evaluated on RS semantic segmentation benchmarks for domain adaptation and domain generalization, our approach achieves mIoU improvements of +9.0% and +3.1% over the Rein baseline, respectively—demonstrating effective suppression of RS-specific corruptions and superior generalization capability.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Parameter-Efficient Fine-Tuning (PEFT) is a technique that allows us to adapt powerful Foundation Models (FMs) to diverse downstream tasks while preserving and unleashing their inherent capabilities. However, we have observed that existing PEFT methods, which are often designed with natural imagery in mind, struggle when applied to Remote Sensing (RS) scenarios. This is primarily due to their inability to handle artifact influences, a problem particularly severe in RS image features. To tackle this challenge, we introduce Earth-Adapter, the first PEFT method specifically designed for RS artifacts conquering. Earth-Adapter introduces a novel Mixture of Frequency Adaptation process that combines a Mixture of Adapter (MoA) with Discrete Fourier Transformation (DFT). By utilizing DFT, Earth-Adapter can decompose features into different frequency components, precisely separating artifacts from original features. The MoA then dynamically assigns weights to each adapter expert, allowing for the combination of features across various frequency domains. These simple-yet-effective approaches enable Earth-Adapter to more efficiently overcome the disturbances caused by artifacts than previous PEFT methods, significantly enhancing the FMs' performance on RS scenarios. Experiments on Domain Adaptation (DA), and Domain Generalization (DG) semantic segmentation benchmarks showcase the Earth-Adapter's effectiveness. Compared with baseline Rein, Earth-Adapter significantly improves 9.0% mIoU in DA and 3.1% mIoU in DG benchmarks. Our code will be released at https://github.com/VisionXLab/Earth-Adapter.

Problem

• Adapts Foundation Models to Remote Sensing scenarios effectively
• Overcomes artifact disturbances in geospatial image features
• Enhances performance in Domain Adaptation and Generalization tasks

Innovation

• Mixture of Frequency Adaptation for RS artifacts
• Combines Mixture of Adapter with Fourier Transform
• Dynamically weights adapters per frequency domain