Multi-Scale Global-Instance Prompt Tuning for Continual Test-Time Adaptation in Medical Image Segmentation

2025-12-15

IEEE International Conference on Bioinformatics and Biomedicine

AI Summary

This work addresses the challenge of continual test-time adaptation (CTTA) in medical image segmentation, where distribution shifts across multi-center data and continuously evolving target domains hinder model performance. To tackle this, we propose the Multi-scale Global-Instance Prompt Tuning (MGIPT) method, which operates without updating backbone parameters or accessing source-domain data. MGIPT synergistically combines Adaptive-scale Instance Prompts (AIP) and Multi-scale Global Prompts (MGP), enhanced by an adaptive optimal scale selection mechanism and a weighted ensemble strategy to mitigate error accumulation and catastrophic forgetting. Extensive experiments demonstrate that MGIPT significantly outperforms existing approaches across multiple medical segmentation benchmarks, achieving stable, efficient, and robust continual domain adaptation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Distribution shift is a common challenge in medical images obtained from different clinical centers, significantly hindering the deployment of pre-trained semantic segmentation models in real-world applications across multiple domains. Continual Test-Time Adaptation (CTTA) has emerged as a promising approach to address cross-domain distribution shifts during continually evolving target domains. Most existing CTTA methods rely on incrementally updating model parameters, which inevitably suffer from error accumulation and catastrophic forgetting, especially in long-term adaptation. Recent prompt-tuning-based works have shown potential to mitigate the two issues above by updating only visual prompts. While these approaches have demonstrated promising performance, several limitations remain: 1) lacking multi-scale prompt diversity, 2) inadequate incorporation of instance-specific knowledge, and 3) risk of privacy leakage. To overcome these limitations, we propose Multi-scale Global-Instance Prompt Tuning (MGIPT), to enhance scale diversity of prompts as well as capture both globaland instance-level knowledge for robust CTTA. Specifically, MGIPT consists of an Adaptive-scale Instance Prompt (AIP) and a Multi-scale Global-level Prompt (MGP). AIP dynamically learns lightweight and instance-specific prompts to mitigate error accumulation with adaptive optimal-scale selection mechanism. MGP captures domain-level knowledge across different scales to ensure robust adaptation with anti-forgetting capabilities. These complementary components are combined through a weighted ensemble approach, enabling effective dual-level adaptation that integrates both global and local information. Extensive experiments on medical image segmentation benchmarks (five optic disc/cup datasets and four polyp datasets) demonstrate that our MGIPT outperforms state-of-the-art methods, achieving robust adaptation across continually changing target domains. Notably, our MGIPT exhibits particularly strong performance in longterm CTTA scenarios, showing great anti-forgetting ability.

Problem

• distribution shift
• continual test-time adaptation
• medical image segmentation
• catastrophic forgetting
• prompt tuning

Innovation

• Prompt Tuning
• Continual Test-Time Adaptation
• Multi-scale Prompt
• Instance-specific Knowledge
• Medical Image Segmentation