Steady Progress Beats Stagnation: Mutual Aid of Foundation and Conventional Models in Mixed Domain Semi-Supervised Medical Image Segmentation

2025-03-21

AI Summary

In semi-supervised medical image segmentation, vision foundation models often suffer from overconfident erroneous predictions and error accumulation in pseudo-labels due to domain shift. To address this, we propose SynFoC, a synergistic training framework that jointly leverages the strengths of foundation models and conventional segmentation models. Our key innovation is a consensus–divergence consistency regularization mechanism: the foundation model generates high-quality initial pseudo-labels, while the conventional model identifies and rectifies its high-confidence mispredictions. SynFoC further incorporates collaborative optimization, pseudo-label purification, and multi-domain adaptation. Evaluated on four public multi-domain benchmarks, SynFoC achieves significant performance gains—e.g., a 10.31% improvement in Dice score on the Prostate dataset. The implementation is publicly available.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Other Foundations of Search & Optimization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large pretrained visual foundation models exhibit impressive general capabilities. However, the extensive prior knowledge inherent in these models can sometimes be a double-edged sword when adapting them to downstream tasks in specific domains. In the context of semi-supervised medical image segmentation with domain shift, foundation models like MedSAM tend to make overconfident predictions, some of which are incorrect. The error accumulation hinders the effective utilization of unlabeled data and limits further improvements. In this paper, we introduce a Synergistic training framework for Foundation and Conventional models (SynFoC) to address the issue. We observe that a conventional model trained from scratch has the ability to correct the high-confidence mispredictions of the foundation model, while the foundation model can supervise it with high-quality pseudo-labels in the early training stages. Furthermore, to enhance the collaborative training effectiveness of both models and promote reliable convergence towards optimization, the consensus-divergence consistency regularization is proposed. We demonstrate the superiority of our method across four public multi-domain datasets. In particular, our method improves the Dice score by 10.31% on the Prostate dataset. Our code is available at https://github.com/MQinghe/SynFoC .

Problem

• Address overconfident predictions in medical image segmentation
• Enhance utilization of unlabeled data with domain shift
• Improve collaboration between foundation and conventional models

Innovation

• Synergistic training framework for Foundation and Conventional models
• Conventional model corrects foundation model mispredictions
• Consensus-divergence consistency regularization enhances collaboration