OOD-SEG: Out-Of-Distribution detection for image SEGmentation with sparse multi-class positive-only annotations

2024-11-14

arXiv.org

AI Summary

Medical and surgical image segmentation faces two key challenges: (1) prohibitive costs of pixel-level annotations, and (2) mainstream methods’ inability to detect out-of-distribution (OOD) pixels—leading to deployment failure under distributional shift. This paper introduces the first OOD-aware segmentation framework designed for sparse multi-class positive-only labeling: it requires only a few positive-class pixel annotations, with no background or negative labels needed. Innovatively, we embed OOD detection directly into the multi-class segmentation pipeline, establishing an annotation-free paradigm for ID/OOD pixel separation under an unlabeled assumption, and propose an OOD-aware cross-validation evaluation strategy tailored to segmentation tasks. Our method integrates pixel-wise OOD detection, sparse positive-sample learning, multi-class positive supervision, and implicit OOD modeling. Evaluated on hyperspectral and RGB surgical images, it significantly improves OOD detection accuracy while preserving in-distribution segmentation performance, demonstrating strong generalization and robustness.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Despite significant advancements, segmentation based on deep neural networks in medical and surgical imaging faces several challenges, two of which we aim to address in this work. First, acquiring complete pixel-level segmentation labels for medical images is time-consuming and requires domain expertise. Second, typical segmentation pipelines cannot detect out-of-distribution (OOD) pixels, leaving them prone to spurious outputs during deployment. In this work, we propose a novel segmentation approach exploiting OOD detection that learns only from sparsely annotated pixels from multiple positive-only classes. These multi-class positive annotations naturally fall within the in-distribution (ID) set. Unlabelled pixels may contain positive classes but also negative ones, including what is typically referred to as emph{background} in standard segmentation formulations. Here, we forgo the need for background annotation and consider these together with any other unseen classes as part of the OOD set. Our framework can integrate, at a pixel-level, any OOD detection approaches designed for classification tasks. To address the lack of existing OOD datasets and established evaluation metric for medical image segmentation, we propose a cross-validation strategy that treats held-out labelled classes as OOD. Extensive experiments on both multi-class hyperspectral and RGB surgical imaging datasets demonstrate the robustness and generalisation capability of our proposed framework.

Problem

• Addresses sparse multi-class positive-only annotations in segmentation
• Eliminates need for background class annotation in medical imaging
• Integrates OOD detection to identify unseen classes and background

Innovation

• Uses positive-unlabelled learning paradigm
• Integrates OOD detection for segmentation
• Eliminates need for background annotations