Cross-device Collaborative Test-time Adaptation with Zeroth-order Optimization and Model Merging

2026-07-03

AI Summary

This work addresses the challenge of deploying test-time adaptation on resource-constrained edge devices, where existing methods relying on backpropagation incur prohibitive memory and computational overhead. To overcome this limitation, the authors propose a cross-device collaborative adaptation framework that integrates zeroth-order optimization (ZOO) with model fusion, enabling edge devices to update models using only forward passes. A novel weight-clipping preprocessing strategy is introduced to substantially reduce the optimization dimensionality and communication redundancy. This approach represents the first synergistic application of ZOO and model fusion for test-time adaptation, demonstrating strong empirical performance on standard benchmarks involving image corruptions and style shifts, while achieving efficient deployment and improved accuracy on edge devices.

Abstract

Test-time adaptation (TTA) mitigates domain shifts by using incoming test data to update a model on the fly. The majority of TTA methods require resource-intensive backpropagation (BP) for model updates, particularly demanding large memory sizes, which makes it infeasible to deploy them on resource-limited devices (e.g., edge devices). To address this issue, we integrate two different techniques, zeroth-order optimization (ZOO) and model merging, under the recently established cross-device collaborative TTA (CDC-TTA) framework, where the system is composed of a mixture of resource-abundant and resource-limited devices, and the model information (e.g., model weights obtained on each device) is shared across the devices. Our method is executable on resource-limited devices by introducing ZOO, which requires only forward processing and bypasses the resource-intensive BP optimization. Concurrently, to mitigate the high-dimensional optimization difficulty caused by the side effect of ZOO, we incorporate model merging of the shared multiple models and set the merge coefficients as the optimization objective, which successfully reduces the optimization dimension. In addition, to enhance the synergistic combination of ZOO and model merging, we propose a unique preprocessing strategy that trims intra-model non-influential weights and reduces the inter-model information redundancy. We empirically confirmed the effectiveness of our method using common corruption and style-transferred image benchmarks.

Problem

• Test-time Adaptation
• Resource-limited Devices
• Domain Shift
• Cross-device Collaboration
• Backpropagation

Innovation

• zeroth-order optimization
• model merging
• test-time adaptation
• cross-device collaboration
• edge devices