A Large-Scale Referring Remote Sensing Image Segmentation Dataset and Benchmark

2025-06-04

AI Summary

Existing remote sensing referring segmentation datasets suffer from low resolution, limited scene diversity, and insufficient category coverage, severely constraining model generalization. To address this, we introduce NWPU-Refer—the first large-scale, highly diverse benchmark for remote sensing referring segmentation—comprising 15,003 high-resolution images spanning over 30 countries and 49,745 annotated objects, supporting single-, multi-, and non-target segmentation. We further propose MRSNet, a dedicated multi-scale network featuring an intra-scale feature interaction module (IFIM) and a hierarchical cross-scale fusion module (HFIM) to jointly model semantic and spatial information. Leveraging a Transformer-CNN hybrid encoder, a multi-scale feature pyramid, and language-vision alignment, MRSNet achieves state-of-the-art mIoU on NWPU-Refer, improving by 5.2% over prior methods with significantly enhanced generalization. Both the dataset and code are publicly released.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Referring Remote Sensing Image Segmentation is a complex and challenging task that integrates the paradigms of computer vision and natural language processing. Existing datasets for RRSIS suffer from critical limitations in resolution, scene diversity, and category coverage, which hinders the generalization and real-world applicability of refer segmentation models. To facilitate the development of this field, we introduce NWPU-Refer, the largest and most diverse RRSIS dataset to date, comprising 15,003 high-resolution images (1024-2048px) spanning 30+ countries with 49,745 annotated targets supporting single-object, multi-object, and non-object segmentation scenarios. Additionally, we propose the Multi-scale Referring Segmentation Network (MRSNet), a novel framework tailored for the unique demands of RRSIS. MRSNet introduces two key innovations: (1) an Intra-scale Feature Interaction Module (IFIM) that captures fine-grained details within each encoder stage, and (2) a Hierarchical Feature Interaction Module (HFIM) to enable seamless cross-scale feature fusion, preserving spatial integrity while enhancing discriminative power. Extensive experiments conducte on the proposed NWPU-Refer dataset demonstrate that MRSNet achieves state-of-the-art performance across multiple evaluation metrics, validating its effectiveness. The dataset and code are publicly available at https://github.com/CVer-Yang/NWPU-Refer.

Problem

• Lack of diverse high-resolution datasets for RRSIS
• Need for improved generalization in refer segmentation models
• Demand for advanced frameworks handling multi-scale feature fusion

Innovation

• Largest diverse RRSIS dataset NWPU-Refer
• Novel MRSNet for RRSIS tasks
• Intra-scale and cross-scale feature fusion