SloMo-Fast: Slow-Momentum and Fast-Adaptive Teachers for Source-Free Continual Test-Time Adaptation

2025-11-23

AI Summary

To address long-term catastrophic forgetting and degraded generalization in source-free continual test-time adaptation (CTTA) under dynamically shifting target domains, this paper proposes SloMo-Fast—a source-free dual-teacher framework. The Slow-Teacher preserves long-term knowledge via momentum-based parameter updates, while the Fast-Teacher rapidly adapts to emerging domain shifts through adaptive optimization. Their synergistic collaboration enables cumulative knowledge integration and unsupervised continual adaptation without accessing source data or class prototypes. We further introduce Cyclic-TTA, a novel benchmark simulating periodic domain shifts to better evaluate temporal robustness. Extensive experiments demonstrate that SloMo-Fast significantly outperforms state-of-the-art methods on Cyclic-TTA and ten established CTTA benchmarks. Results validate its strong robustness to both evolving and revisiting domains, sustained adaptability, and superior generalization capability under strict source-free constraints.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Continual Test-Time Adaptation (CTTA) is crucial for deploying models in real-world applications with unseen, evolving target domains. Existing CTTA methods, however, often rely on source data or prototypes, limiting their applicability in privacy-sensitive and resource-constrained settings. Additionally, these methods suffer from long-term forgetting, which degrades performance on previously encountered domains as target domains shift. To address these challenges, we propose SloMo-Fast, a source-free, dual-teacher CTTA framework designed for enhanced adaptability and generalization. It includes two complementary teachers: the Slow-Teacher, which exhibits slow forgetting and retains long-term knowledge of previously encountered domains to ensure robust generalization, and the Fast-Teacher rapidly adapts to new domains while accumulating and integrating knowledge across them. This framework preserves knowledge of past domains and adapts efficiently to new ones. We also introduce Cyclic Test-Time Adaptation (Cyclic-TTA), a novel CTTA benchmark that simulates recurring domain shifts. Our extensive experiments demonstrate that SloMo-Fast consistently outperforms state-of-the-art methods across Cyclic-TTA, as well as ten other CTTA settings, highlighting its ability to both adapt and generalize across evolving and revisited domains.

Problem

• Addresses source-free continual adaptation to evolving domains
• Mitigates long-term forgetting in privacy-sensitive environments
• Enhances model generalization across recurring domain shifts

Innovation

• Slow-momentum teacher prevents long-term forgetting
• Fast-adaptive teacher enables rapid domain adaptation
• Dual-teacher framework operates without source data