On-demand Test-time Adaptation for Edge Devices

2025-05-02

AI Summary

To address the high memory overhead, energy consumption, and limited practicality of continual test-time adaptation (CTTA) methods on edge devices, this paper proposes on-demand test-time adaptation (OD-TTA): model updates are triggered only when a lightweight domain shift detector identifies significant distributional shifts. Our key contributions are: (1) a feature-statistics-based lightweight online domain shift detection mechanism; (2) a source-model dynamic retrieval module that avoids redundant parameter loading; and (3) a decoupled BatchNorm parameter update strategy enabling efficient local adaptation. Extensive experiments across multiple benchmark datasets demonstrate that OD-TTA maintains or even improves accuracy while reducing memory footprint by 37–62% and inference energy consumption by 41–58%, significantly enhancing deployment feasibility and robustness in resource-constrained edge environments.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Adversarial Attacks & Robustness
• Application Domains: Internet of Things, Sensor Networks & Smart Cities

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

Continual Test-time adaptation (CTTA) continuously adapts the deployed model on every incoming batch of data. While achieving optimal accuracy, existing CTTA approaches present poor real-world applicability on resource-constrained edge devices, due to the substantial memory overhead and energy consumption. In this work, we first introduce a novel paradigm -- on-demand TTA -- which triggers adaptation only when a significant domain shift is detected. Then, we present OD-TTA, an on-demand TTA framework for accurate and efficient adaptation on edge devices. OD-TTA comprises three innovative techniques: 1) a lightweight domain shift detection mechanism to activate TTA only when it is needed, drastically reducing the overall computation overhead, 2) a source domain selection module that chooses an appropriate source model for adaptation, ensuring high and robust accuracy, 3) a decoupled Batch Normalization (BN) update scheme to enable memory-efficient adaptation with small batch sizes. Extensive experiments show that OD-TTA achieves comparable and even better performance while reducing the energy and computation overhead remarkably, making TTA a practical reality.

Problem

• Reducing memory and energy overhead in continual test-time adaptation for edge devices
• Triggering adaptation only when significant domain shifts are detected
• Enabling efficient and accurate adaptation with small batch sizes

Innovation

• Lightweight domain shift detection for on-demand TTA
• Source domain selection module for robust accuracy
• Decoupled BN update scheme for memory efficiency