Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancement

2026-01-05

arXiv.org

AI Summary

This work addresses the significant performance degradation of the Segment Anything Model (SAM) on low-quality images, which struggles with the diverse image degradations encountered in real-world scenarios. To mitigate this limitation, the authors propose a Degradation-Aware Adaptive Enhancement (DAE) mechanism that decouples the restoration process into two stages: degradation-level prediction and perception-aware reconstruction. By integrating generative latent-space enhancement, Feature Distribution Alignment (FDA), and Channel Replication Expansion (CRE), the method improves the compatibility between diffusion-based restoration and segmentation frameworks. Extensive experiments demonstrate that the proposed approach substantially enhances segmentation robustness across various complex degradations while preserving strong generalization on clean images and exhibiting notable adaptability to unseen degradation types.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Segment Anything Models (SAMs), known for their exceptional zero-shot segmentation performance, have garnered significant attention in the research community. Nevertheless, their performance drops significantly on severely degraded, low-quality images, limiting their effectiveness in real-world scenarios. To address this, we propose GleSAM++, which utilizes Generative Latent space Enhancement to boost robustness on low-quality images, thus enabling generalization across various image qualities. Additionally, to improve compatibility between the pre-trained diffusion model and the segmentation framework, we introduce two techniques, i.e., Feature Distribution Alignment (FDA) and Channel Replication and Expansion (CRE). However, the above components lack explicit guidance regarding the degree of degradation. The model is forced to implicitly fit a complex noise distribution that spans conditions from mild noise to severe artifacts, which substantially increases the learning burden and leads to suboptimal reconstructions. To address this issue, we further introduce a Degradation-aware Adaptive Enhancement (DAE) mechanism. The key principle of DAE is to decouple the reconstruction process for arbitrary-quality features into two stages: degradation-level prediction and degradation-aware reconstruction. Our method can be applied to pre-trained SAM and SAM2 with only minimal additional learnable parameters, allowing for efficient optimization. Extensive experiments demonstrate that GleSAM++ significantly improves segmentation robustness on complex degradations while maintaining generalization to clear images. Furthermore, GleSAM++ also performs well on unseen degradations, underscoring the versatility of our approach and dataset.

Problem

• image segmentation
• low-quality images
• degradation robustness
• zero-shot segmentation
• image quality generalization

Innovation

• Generative Latent Space Enhancement
• Degradation-aware Adaptive Enhancement
• Feature Distribution Alignment
• Channel Replication and Expansion
• Any-Quality Segmentation