SPEGC: Continual Test-Time Adaptation via Semantic-Prompt-Enhanced Graph Clustering for Medical Image Segmentation

2026-03-11

AI Summary

This work addresses the performance degradation and error accumulation in medical image segmentation caused by continuously shifting test domains. To this end, we propose a semantic prompt–enhanced continual test-time adaptation (CTTA) method that decouples shared and heterogeneous semantic prompts to enrich local features. Our approach integrates a differentiable graph clustering solver based on optimal transport to adjust decision boundaries at the cluster level, guided by high-order structural consistency. This design effectively mitigates noise induced by domain shift and prevents catastrophic forgetting. Extensive experiments demonstrate that our method significantly outperforms existing CTTA approaches on two medical image segmentation benchmarks, achieving substantial improvements in both robustness and accuracy under cross-domain settings.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In medical image segmentation tasks, the domain gap caused by the difference in data collection between training and testing data seriously hinders the deployment of pre-trained models in clinical practice. Continual Test-Time Adaptation (CTTA) aims to enable pre-trained models to adapt to continuously changing unlabeled domains, providing an effective approach to solving this problem. However, existing CTTA methods often rely on unreliable supervisory signals, igniting a self-reinforcing cycle of error accumulation that culminates in catastrophic performance degradation. To overcome these challenges, we propose a CTTA via Semantic-Prompt-Enhanced Graph Clustering (SPEGC) for medical image segmentation. First, we design a semantic prompt feature enhancement mechanism that utilizes decoupled commonality and heterogeneity prompt pools to inject global contextual information into local features, alleviating their susceptibility to noise interference under domain shift. Second, based on these enhanced features, we design a differentiable graph clustering solver. This solver reframes global edge sparsification as an optimal transport problem, allowing it to distill a raw similarity matrix into a refined and high-order structural representation in an end-to-end manner. Finally, this robust structural representation is used to guide model adaptation, ensuring predictions are consistent at a cluster-level and dynamically adjusting decision boundaries. Extensive experiments demonstrate that SPEGC outperforms other state-of-the-art CTTA methods on two medical image segmentation benchmarks. The source code is available at https://github.com/Jwei-Z/SPEGC-for-MIS.

Problem

• domain gap
• continual test-time adaptation
• medical image segmentation
• error accumulation
• unlabeled domains

Innovation

• Continual Test-Time Adaptation
• Semantic Prompt
• Graph Clustering
• Optimal Transport
• Medical Image Segmentation