A3-TTA: Adaptive Anchor Alignment Test-Time Adaptation for Image Segmentation

2025-12-22

IEEE Transactions on Image Processing

AI Summary

This work addresses the performance degradation of image segmentation models under test-time domain shift when source data are inaccessible and retraining is infeasible. The authors propose an anchor-guided test-time adaptation method that leverages a distributional proximity assumption to identify high-confidence target samples as anchors via a class-compactness density metric. Reliable pseudo-labels are generated by integrating semantic consistency constraints with boundary-aware entropy minimization. To mitigate pseudo-label noise and catastrophic forgetting, an adaptive exponential moving average (EMA) strategy is introduced for model updating. Evaluated on multi-domain segmentation tasks in both medical and natural images, the method improves the average Dice score by 10.40–17.68 percentage points over the source model, substantially outperforming existing test-time adaptation approaches, and demonstrates strong resilience to catastrophic forgetting in continuous adaptation scenarios.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Segmentation
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Test-Time Adaptation (TTA) offers a practical solution for deploying image segmentation models under domain shift without accessing source data or retraining. Among existing TTA strategies, pseudo-label-based methods have shown promising performance. However, they often rely on perturbation-ensemble heuristics (e.g., dropout sampling, test-time augmentation, Gaussian noise), which lack distributional grounding and yield unstable training signals. This can trigger error accumulation and catastrophic forgetting during adaptation. To address this, we propose A3-TTA, a TTA framework that constructs reliable pseudo-labels through anchor-guided supervision. Specifically, we identify well-predicted target domain images using a class compact density metric, under the assumption that confident predictions imply distributional proximity to the source domain. These anchors serve as stable references to guide pseudo-label generation, which is further regularized via semantic consistency and boundary-aware entropy minimization. Additionally, we introduce a self-adaptive exponential moving average strategy to mitigate label noise and stabilize model update during adaptation. Evaluated on both multi-domain medical images (heart structure and prostate segmentation) and natural images, A3-TTA significantly improves average Dice scores by 10.40 to 17.68 percentage points compared to the source model, outperforming several state-of-the-art TTA methods under different segmentation model architectures. A3-TTA also excels in continual TTA, maintaining high performance across sequential target domains with strong anti-forgetting ability. The code will be made publicly available at https://github.com/HiLab-git/A3-TTA

Problem

• Test-Time Adaptation
• Domain Shift
• Image Segmentation
• Pseudo-Labeling
• Catastrophic Forgetting

Innovation

• Test-Time Adaptation
• Anchor-Guided Supervision
• Pseudo-Label Refinement
• Domain Shift
• Continual Adaptation