Semantic-Driven Scale and Spatial Selection for Efficient Cross-Modal Alignment in Referring Remote Sensing Image Segmentation

2026-06-29

AI Summary

Existing referring expression segmentation methods for remote sensing rely on full-parameter fine-tuning, which incurs high computational costs and risks degrading the generalization capability of pretrained models. Meanwhile, current parameter-efficient tuning approaches struggle to model cross-modal dependencies and bridge the domain gap between natural and aerial images. To address these limitations, this work proposes the S4ECA framework, featuring a dual-encoder adapter architecture: a text adapter generates high-level, learnable linguistic proxies, while a visual adapter integrates multi-scale dense features. A novel semantic-driven scale and spatial selection mechanism enables language-guided dynamic region focusing. By updating only 2.4% of the backbone parameters, S4ECA achieves state-of-the-art performance on both RRSIS-D and RefSegRS benchmarks, significantly enhancing segmentation accuracy and computational efficiency in complex aerial scenes.

Abstract

Referring Remote Sensing Image Segmentation (RRSIS) seeks to localize and segment the target object or region specified by a natural language expression in a remote sensing image. While existing RRSIS models have benefited from large-scale foundation models, they predominantly rely on full fine-tuning. These approaches are computationally intensive and may weaken the generalization ability of pre-trained models, as extensive fine-tuning on significantly smaller downstream datasets can distort the well-structured feature representations learned during large-scale pre-training. Although Parameter-Efficient Tuning (PET) offers a potential alternative, existing PET frameworks primarily focus on single-modal optimization, failing to capture the complex cross-modal dependencies required for multimodal reasoning, while simultaneously struggling to bridge the substantial domain gap between natural scenes and aerial imagery. To address these limitations, we propose a novel framework, Semantic-driven Scale and Spatial Selection for Efficient Cross-modal Alignment (S4ECA), which enables effective and efficient cross-modal interaction through parameter-efficient adaptation. Specifically, we design a dual-encoder adapter architecture. The textual adapter employs learnable queries to distill highly semantic language proxies from word-level embeddings, facilitating early grounding. Simultaneously, the visual adapter refines hierarchical feature representations through a multi-scale dense extractor, followed by a language-guided scale and spatial selection mechanism that dynamically emphasizes relevant visual contexts, ensuring precise cross-modal alignment. By updating only 2.4% of the backbone parameters, our proposed model achieves state-of-the-art performance on the RRSIS-D and RefSegRS datasets, demonstrating superior efficiency and precision in complex aerial scenarios.

Problem

• Referring Remote Sensing Image Segmentation
• Cross-Modal Alignment
• Parameter-Efficient Tuning
• Domain Gap
• Multimodal Reasoning

Innovation

• Parameter-Efficient Tuning
• Cross-Modal Alignment
• Referring Remote Sensing Image Segmentation
• Semantic-Driven Selection
• Dual-Encoder Adapter