Towards High-Quality Image Segmentation: Improving Topology Accuracy by Penalizing Neighbor Pixels

2026-03-19

AI Summary

This work addresses the degradation of segmentation quality and reliability in quantitative analysis caused by topological errors—such as incorrect numbers or structures of connected components—in existing image segmentation models. To this end, we propose SCNP, a lightweight, general-purpose, and easily integrable topological refinement mechanism that enhances global topological accuracy by prioritizing local consistency. Specifically, SCNP penalizes the logits of each pixel relative to the worst-classified pixel within its same-class neighborhood, thereby guiding the model to correct local inconsistencies that often lead to topological defects. Notably, our method is agnostic to object morphology, imaging modality, segmentation architecture, and loss function, overcoming limitations of prior approaches in complexity, compatibility, and applicability. Experiments demonstrate that SCNP consistently improves topological accuracy across 13 diverse datasets and seamlessly integrates with three major segmentation frameworks.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Standard deep learning models for image segmentation cannot guarantee topology accuracy, failing to preserve the correct number of connected components or structures. This, in turn, affects the quality of the segmentations and compromises the reliability of the subsequent quantification analyses. Previous works have proposed to enhance topology accuracy with specialized frameworks, architectures, and loss functions. However, these methods are often cumbersome to integrate into existing training pipelines, they are computationally very expensive, or they are restricted to structures with tubular morphology. We present SCNP, an efficient method that improves topology accuracy by penalizing the logits with their poorest-classified neighbor, forcing the model to improve the prediction at the pixels' neighbors before allowing it to improve the pixels themselves. We show the effectiveness of SCNP across 13 datasets, covering different structure morphologies and image modalities, and integrate it into three frameworks for semantic and instance segmentation. Additionally, we show that SCNP can be integrated into several loss functions, making them improve topology accuracy. Our code can be found at https://jmlipman.github.io/SCNP-SameClassNeighborPenalization.

Problem

• image segmentation
• topology accuracy
• connected components
• segmentation quality
• quantification analysis

Innovation

• topology accuracy
• image segmentation
• neighbor penalization
• SCNP
• deep learning