From Few to More: Scribble-based Medical Image Segmentation via Masked Context Modeling and Continuous Pseudo Labels

2024-08-23

arXiv.org

AI Summary

To address performance degradation in weakly supervised medical image segmentation caused by sparse scribble annotations, this paper proposes MaCo, a “from-few-to-many” progressive learning framework. Methodologically, we introduce Mask Context Modeling (MCM), a novel attention mechanism that jointly leverages distance maps and an exponential decay function to generate continuous pseudo-labels (CPLs) with pixel-wise semantic confidence—replacing error-prone hard pseudo-labels. Additionally, we incorporate a self-supervised context consistency constraint, eliminating reliance on auxiliary tasks. This enables robust, continuous modeling of pixel-level semantic confidence. Evaluated on three public medical imaging benchmarks, MaCo consistently outperforms existing scribble-supervised methods, establishing new state-of-the-art results and significantly enhancing model adaptability to annotation sparsity.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Segmentation
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Scribble-based weakly supervised segmentation techniques offer comparable performance to fully supervised methods while significantly reducing annotation costs, making them an appealing alternative. Existing methods often rely on auxiliary tasks to enforce semantic consistency and use hard pseudo labels for supervision. However, these methods often overlook the unique requirements of models trained with sparse annotations. Since the model must predict pixel-wise segmentation maps with limited annotations, the ability to handle varying levels of annotation richness is critical. In this paper, we adopt the principle of `from few to more' and propose MaCo, a weakly supervised framework designed for medical image segmentation. MaCo employs masked context modeling (MCM) and continuous pseudo labels (CPL). MCM uses an attention-based masking strategy to disrupt the input image, compelling the model's predictions to remain consistent with those of the original image. CPL converts scribble annotations into continuous pixel-wise labels by applying an exponential decay function to distance maps, resulting in continuous maps that represent the confidence of each pixel belonging to a specific category, rather than using hard pseudo labels. We evaluate MaCo against other weakly supervised methods using three public datasets. The results indicate that MaCo outperforms competing methods across all datasets, setting a new record in weakly supervised medical image segmentation.

Problem

• Improves scribble-based medical image segmentation accuracy
• Addresses sparse annotation challenges in segmentation models
• Enhances semantic consistency without hard pseudo labels

Innovation

• Uses Masked Context Modeling for semantic consistency
• Applies Continuous Pseudo Labels from scribble annotations
• Leverages attention-based masking for robust predictions