MFM-DA: Instance-Aware Adaptor and Hierarchical Alignment for Efficient Domain Adaptation in Medical Foundation Models

2025-03-02

AI Summary

Medical foundation models (MFMs) suffer substantial performance degradation during cross-domain deployment due to domain shift—especially when labeled target-domain data is scarce. This work proposes a few-shot unsupervised domain adaptation framework that enables efficient transfer using only a small number of unlabeled target-domain images. Our method jointly leverages (1) a dynamic instance-aware adapter and a distribution-direction loss to steer DDPM-style generative adaptation, and (2) a channel-spatial aligned LoRA module for fine-grained, hierarchical feature alignment within MFMs. Evaluated on optic cup and optic disc segmentation, the approach significantly outperforms existing state-of-the-art methods, effectively bridging the performance gap between source-domain fine-tuned MFMs and target-domain deployment. It enhances both clinical generalizability and practical deployability of MFMs in real-world medical imaging scenarios.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Application Domains: Mobility, Driving & Flight

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Medical Foundation Models (MFMs), trained on large-scale datasets, have demonstrated superior performance across various tasks. However, these models still struggle with domain gaps in practical applications. Specifically, even after fine-tuning on source-domain data, task-adapted foundation models often perform poorly in the target domain. To address this challenge, we propose a few-shot unsupervised domain adaptation (UDA) framework for MFMs, named MFM-DA, which only leverages a limited number of unlabeled target-domain images. Our approach begins by training a Denoising Diffusion Probabilistic Model (DDPM), which is then adapted to the target domain using a proposed dynamic instance-aware adaptor and a distribution direction loss, enabling the DDPM to translate source-domain images into the target domain style. The adapted images are subsequently processed through the MFM, where we introduce a designed channel-spatial alignment Low-Rank Adaptation (LoRA) to ensure effective feature alignment. Extensive experiments on optic cup and disc segmentation tasks demonstrate that MFM-DA outperforms state-of-the-art methods. Our work provides a practical solution to the domain gap issue in real-world MFM deployment. Code will be available at here.

Problem

• Addresses domain gaps in Medical Foundation Models (MFMs).
• Proposes few-shot unsupervised domain adaptation (UDA) for MFMs.
• Enhances feature alignment with dynamic instance-aware adaptor.

Innovation

• Dynamic instance-aware adaptor for domain adaptation
• Distribution direction loss for style translation
• Channel-spatial alignment LoRA for feature alignment