DINO-Mix: Distilling Foundational Knowledge with Cross-Domain CutMix for Semi-supervised Class-imbalanced Medical Image Segmentation

2026-02-08

AI Summary

This work addresses the confirmation bias induced by severe class imbalance in semi-supervised medical image segmentation, particularly the failure to accurately segment minority classes under limited annotation. To mitigate this issue, the authors propose a multi-level “look-outward” framework that leverages cross-domain knowledge distillation from an external vision foundation model (DINOv3) and introduces a progressive imbalance-aware CutMix (PIC) strategy to dynamically generate unbiased supervision signals. By moving beyond the conventional reliance on internal information within the target dataset alone, the method significantly enhances segmentation performance for underrepresented classes. State-of-the-art results are achieved on challenging, highly imbalanced benchmarks such as Synapse and AMOS, demonstrating the effectiveness of the proposed approach in real-world medical imaging scenarios with scarce labels.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Semi-supervised learning (SSL) has emerged as a critical paradigm for medical image segmentation, mitigating the immense cost of dense annotations. However, prevailing SSL frameworks are fundamentally"inward-looking", recycling information and biases solely from within the target dataset. This design triggers a vicious cycle of confirmation bias under class imbalance, leading to the catastrophic failure to recognize minority classes. To dismantle this systemic issue, we propose a paradigm shift to a multi-level"outward-looking"framework. Our primary innovation is Foundational Knowledge Distillation (FKD), which looks outward beyond the confines of medical imaging by introducing a pre-trained visual foundation model, DINOv3, as an unbiased external semantic teacher. Instead of trusting the student's biased high confidence, our method distills knowledge from DINOv3's robust understanding of high semantic uniqueness, providing a stable, cross-domain supervisory signal that anchors the learning of minority classes. To complement this core strategy, we further look outward within the data by proposing Progressive Imbalance-aware CutMix (PIC), which creates a dynamic curriculum that adaptively forces the model to focus on minority classes in both labeled and unlabeled subsets. This layered strategy forms our framework, DINO-Mix, which breaks the vicious cycle of bias and achieves remarkable performance on challenging semi-supervised class-imbalanced medical image segmentation benchmarks Synapse and AMOS.

Problem

• semi-supervised learning
• class-imbalanced segmentation
• medical image segmentation
• confirmation bias
• minority classes

Innovation

• Foundational Knowledge Distillation
• Cross-domain Supervision
• Class-imbalanced Segmentation
• Semi-supervised Learning
• Progressive Imbalance-aware CutMix