Tell2Adapt: A Unified Framework for Source Free Unsupervised Domain Adaptation via Vision Foundation Model

2026-03-05

AI Summary

This work addresses the challenge of applying source-free unsupervised domain adaptation (SFUDA) to complex medical imaging scenarios involving multiple modalities and anatomical structures, which has hindered clinical deployment. We propose the first unified SFUDA framework for multi-modal, multi-organ segmentation, leveraging the general-purpose representations of vision foundation models (VFMs). Our approach introduces context-aware prompt regularization (CAPR) to generate high-quality pseudo-labels and incorporates an anatomy-aware visual plausibility refinement (VPR) mechanism to enhance prediction consistency with anatomical priors. The framework enables efficient adaptation of lightweight student models without access to source data. Extensive experiments across 10 domain transfer settings and 22 anatomical structures demonstrate substantial performance gains over existing methods, establishing a new state of the art in SFUDA for medical image segmentation.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Source Free Unsupervised Domain Adaptation (SFUDA) is critical for deploying deep learning models across diverse clinical settings. However, existing methods are typically designed for low-gap, specific domain shifts and cannot generalize into a unified, multi-modalities, and multi-target framework, which presents a major barrier to real-world application. To overcome this issue, we introduce Tell2Adapt, a novel SFUDA framework that harnesses the vast, generalizable knowledge of the Vision Foundation Model (VFM). Our approach ensures high-fidelity VFM prompts through Context-Aware Prompts Regularization (CAPR), which robustly translates varied text prompts into canonical instructions. This enables the generation of high-quality pseudo-labels for efficiently adapting the lightweight student model to target domain. To guarantee clinical reliability, the framework incorporates Visual Plausibility Refinement (VPR), which leverages the VFM's anatomical knowledge to re-ground the adapted model's predictions in target image's low-level visual features, effectively removing noise and false positives. We conduct one of the most extensive SFUDA evaluations to date, validating our framework across 10 domain adaptation directions and 22 anatomical targets, including brain, cardiac, polyp, and abdominal targets. Our results demonstrate that Tell2Adapt consistently outperforms existing approaches, achieving SOTA for a unified SFUDA framework in medical image segmentation. Code are avaliable at https://github.com/derekshiii/Tell2Adapt.

Problem

• Source Free Unsupervised Domain Adaptation
• Domain Generalization
• Medical Image Segmentation
• Multi-modality
• Multi-target

Innovation

• Source-Free Unsupervised Domain Adaptation
• Vision Foundation Model
• Context-Aware Prompts Regularization
• Visual Plausibility Refinement
• Medical Image Segmentation