SegAssess: Panoramic quality mapping for robust and transferable unsupervised segmentation assessment

2025-09-01

AI Summary

To address the challenge of evaluating unsupervised segmentation in remote sensing imagery—where ground-truth labels are unavailable—this paper proposes Panoptic Quality Mapping (PQM), a novel paradigm that reformulates segmentation quality assessment as a pixel-wise four-class panoptic segmentation task (TP/FP/TN/FN), enabling fine-grained, comprehensive, and transferable quality map generation for the first time. Methodologically, we design an edge-guided compression branch and an aggregated semantic filtering module, and introduce an enhanced hybrid sampling strategy. Building upon an improved Segment Anything Model (SAM), our approach fuses multi-source mask prompts with cross-attention features. Evaluated on 32 remote sensing datasets, PQM achieves state-of-the-art performance, significantly enhancing cross-domain robustness and zero-shot transferability. The code is publicly available.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

High-quality image segmentation is fundamental to pixel-level geospatial analysis in remote sensing, necessitating robust segmentation quality assessment (SQA), particularly in unsupervised settings lacking ground truth. Although recent deep learning (DL) based unsupervised SQA methods show potential, they often suffer from coarse evaluation granularity, incomplete assessments, and poor transferability. To overcome these limitations, this paper introduces Panoramic Quality Mapping (PQM) as a new paradigm for comprehensive, pixel-wise SQA, and presents SegAssess, a novel deep learning framework realizing this approach. SegAssess distinctively formulates SQA as a fine-grained, four-class panoramic segmentation task, classifying pixels within a segmentation mask under evaluation into true positive (TP), false positive (FP), true negative (TN), and false negative (FN) categories, thereby generating a complete quality map. Leveraging an enhanced Segment Anything Model (SAM) architecture, SegAssess uniquely employs the input mask as a prompt for effective feature integration via cross-attention. Key innovations include an Edge Guided Compaction (EGC) branch with an Aggregated Semantic Filter (ASF) module to refine predictions near challenging object edges, and an Augmented Mixup Sampling (AMS) training strategy integrating multi-source masks to significantly boost cross-domain robustness and zero-shot transferability. Comprehensive experiments across 32 datasets derived from 6 sources demonstrate that SegAssess achieves state-of-the-art (SOTA) performance and exhibits remarkable zero-shot transferability to unseen masks, establishing PQM via SegAssess as a robust and transferable solution for unsupervised SQA. The code is available at https://github.com/Yangbn97/SegAssess.

Problem

• Assesses segmentation quality without ground truth data
• Overcomes coarse granularity and poor transferability in SQA
• Provides pixel-level quality mapping for remote sensing images

Innovation

• Pixel-wise four-class panoramic segmentation task
• Enhanced SAM with cross-attention mask prompting
• Edge Guided Compaction and Augmented Mixup Sampling