SNAP: Low-Latency Test-Time Adaptation with Sparse Updates

2025-11-19

AI Summary

To address the high computational overhead and latency of test-time adaptation (TTA) on edge devices caused by frequent model updates, this paper proposes SNAP, a sparse adaptive framework. Its core contributions are: (1) a class- and domain-aware representative memory module that preserves discriminative knowledge using only 1% of test samples; and (2) inference-level batch-aware memory normalization, which dynamically calibrates feature statistics to enhance robustness under distribution shifts. SNAP seamlessly integrates five mainstream TTA algorithms and enables efficient online adaptation. Experiments demonstrate that, under dynamic distribution shifts, SNAP reduces adaptation latency by up to 93.12%, incurs ≤3.3% accuracy degradation, and maintains stable performance across sparse update rates ranging from 1% to 50%. These results significantly improve efficiency and practicality for edge-deployed TTA.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Test-Time Adaptation (TTA) adjusts models using unlabeled test data to handle dynamic distribution shifts. However, existing methods rely on frequent adaptation and high computational cost, making them unsuitable for resource-constrained edge environments. To address this, we propose SNAP, a sparse TTA framework that reduces adaptation frequency and data usage while preserving accuracy. SNAP maintains competitive accuracy even when adapting based on only 1% of the incoming data stream, demonstrating its robustness under infrequent updates. Our method introduces two key components: (i) Class and Domain Representative Memory (CnDRM), which identifies and stores a small set of samples that are representative of both class and domain characteristics to support efficient adaptation with limited data; and (ii) Inference-only Batch-aware Memory Normalization (IoBMN), which dynamically adjusts normalization statistics at inference time by leveraging these representative samples, enabling efficient alignment to shifting target domains. Integrated with five state-of-the-art TTA algorithms, SNAP reduces latency by up to 93.12%, while keeping the accuracy drop below 3.3%, even across adaptation rates ranging from 1% to 50%. This demonstrates its strong potential for practical use on edge devices serving latency-sensitive applications. The source code is available at https://github.com/chahh9808/SNAP.

Problem

• Reducing adaptation frequency for test-time models on edge devices
• Minimizing computational cost while maintaining competitive accuracy
• Enabling efficient adaptation with sparse data usage in dynamic environments

Innovation

• Sparse TTA framework reduces adaptation frequency
• CnDRM stores representative samples for efficient adaptation
• IoBMN adjusts normalization statistics at inference time