Adapting Foundation Models for Few-Shot Medical Image Segmentation: Actively and Sequentially

2025-02-03

AI Summary

To address the challenges of large source-target domain discrepancies and severe label scarcity in few-shot medical image segmentation, this paper proposes an active, sequential auxiliary dataset selection framework—first modeling few-shot domain adaptation as a multi-armed bandit problem. We design a reward function evaluable with a single-round fine-tuning step, eliminating negative transfer induced by heuristic selection. By unifying foundation model adaptation, few-shot domain adaptation, active learning, and reinforcement learning, our method enables dynamic scheduling of auxiliary data and feature alignment across domains. On multi-center MRI and CT segmentation benchmarks, it achieves average Dice score improvements of 27.75% and 7.52%, respectively, substantially outperforming existing state-of-the-art methods. The core contribution lies in reformulating domain adaptation as a learnable sequential decision-making problem, thereby ensuring both efficiency and robust generalization under extreme label scarcity.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advances in foundation models have brought promising results in computer vision, including medical image segmentation. Fine-tuning foundation models on specific low-resource medical tasks has become a standard practice. However, ensuring reliable and robust model adaptation when the target task has a large domain gap and few annotated samples remains a challenge. Previous few-shot domain adaptation (FSDA) methods seek to bridge the distribution gap between source and target domains by utilizing auxiliary data. The selection and scheduling of auxiliaries are often based on heuristics, which can easily cause negative transfer. In this work, we propose an Active and Sequential domain AdaPtation (ASAP) framework for dynamic auxiliary dataset selection in FSDA. We formulate FSDA as a multi-armed bandit problem and derive an efficient reward function to prioritize training on auxiliary datasets that align closely with the target task, through a single-round fine-tuning. Empirical validation on diverse medical segmentation datasets demonstrates that our method achieves favorable segmentation performance, significantly outperforming the state-of-the-art FSDA methods, achieving an average gain of 27.75% on MRI and 7.52% on CT datasets in Dice score. Code is available at the git repository: https://github.com/techicoco/ASAP.

Problem

• Few-shot Learning
• Medical Image Segmentation
• Adaptation Gap

Innovation

• ASAP Framework
• Smart Data Selection
• Medical Image Segmentation