Learn Faster and Remember More: Balancing Exploration and Exploitation for Continual Test-time Adaptation

2025-08-18

AI Summary

In continual test-time adaptation (CTTA), existing methods struggle to balance exploration of novel domains with exploitation of historical knowledge: they rely solely on output-layer adjustments, ignoring shallow feature shifts and thus achieving inefficient adaptation; moreover, single-model architectures suffer from catastrophic forgetting, impairing long-term generalization. To address this, we propose an exploration-exploitation synergy framework: (1) Multi-level Consistency Regularization (MCR) enforces teacher-student consistency constraints at intermediate network layers, enhancing sensitivity to shallow domain shifts and accelerating adaptation; (2) Complementary Anchor Replay (CAR) mitigates forgetting by replaying semantically diverse historical anchor samples. Integrated within the Mean Teacher framework, MCR and CAR jointly optimize adaptation dynamics. Extensive experiments on multiple CTTA benchmarks demonstrate substantial improvements over state-of-the-art methods, achieving both rapid domain adaptation and robust retention of historical knowledge.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Continual Test-Time Adaptation (CTTA) aims to adapt a source pre-trained model to continually changing target domains during inference. As a fundamental principle, an ideal CTTA method should rapidly adapt to new domains (exploration) while retaining and exploiting knowledge from previously encountered domains to handle similar domains in the future. Despite significant advances, balancing exploration and exploitation in CTTA is still challenging: 1) Existing methods focus on adjusting predictions based on deep-layer outputs of neural networks. However, domain shifts typically affect shallow features, which are inefficient to be adjusted from deep predictions, leading to dilatory exploration; 2) A single model inevitably forgets knowledge of previous domains during the exploration, making it incapable of exploiting historical knowledge to handle similar future domains. To address these challenges, this paper proposes a mean teacher framework that strikes an appropriate Balance between Exploration and Exploitation (BEE) during the CTTA process. For the former challenge, we introduce a Multi-level Consistency Regularization (MCR) loss that aligns the intermediate features of the student and teacher models, accelerating adaptation to the current domain. For the latter challenge, we employ a Complementary Anchor Replay (CAR) mechanism to reuse historical checkpoints (anchors), recovering complementary knowledge for diverse domains. Experiments show that our method significantly outperforms state-of-the-art methods on several benchmarks, demonstrating its effectiveness for CTTA tasks.

Problem

• Balancing exploration and exploitation in Continual Test-Time Adaptation
• Addressing domain shifts affecting shallow neural network features
• Preventing knowledge forgetting during adaptation to new domains

Innovation

• Mean teacher framework balances exploration and exploitation
• Multi-level Consistency Regularization aligns intermediate features
• Complementary Anchor Replay reuses historical checkpoints