Decentralizing Test-time Adaptation under Heterogeneous Data Streams

2024-11-16

arXiv.org

AI Summary

In real-world deployments, deep learning models often suffer significant performance degradation due to train-test distribution shifts—particularly complex, mixed shifts arising from heterogeneous multi-source data streams. Existing test-time adaptation (TTA) methods lack robustness under label-free, online, and multi-domain coexistence settings. To address this, we propose FreDA, a frequency-driven decentralized adaptation framework—the first TTA approach incorporating Fourier transforms. FreDA models intrinsic frequency structures of data in the frequency domain to achieve local homogenization, and integrates decentralized parameter updates with frequency-domain customized augmentation, thereby abandoning the conventional assumption of a global unified objective. Evaluated on heterogeneous streaming data—including image corruptions, natural scenes, and medical imaging—FreDA consistently outperforms state-of-the-art TTA methods, delivering substantial improvements in both generalization and robustness.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

While Test-Time Adaptation (TTA) has shown promise in addressing distribution shifts between training and testing data, its effectiveness diminishes with heterogeneous data streams due to uniform target estimation. As previous attempts merely stabilize model fine-tuning over time to handle continually changing environments, they fundamentally assume a homogeneous target domain at any moment, leaving the intrinsic real-world data heterogeneity unresolved. This paper delves into TTA under heterogeneous data streams, moving beyond current model-centric limitations. By revisiting TTA from a data-centric perspective, we discover that decomposing samples into Fourier space facilitates an accurate data separation across different frequency levels. Drawing from this insight, we propose a novel Frequency-based Decentralized Adaptation (FreDA) framework, which transitions data from globally heterogeneous to locally homogeneous in Fourier space and employs decentralized adaptation to manage diverse distribution shifts.Interestingly, we devise a novel Fourier-based augmentation strategy to assist in decentralizing adaptation, which individually enhances sample quality for capturing each type of distribution shifts. Extensive experiments across various settings (corrupted, natural, and medical environments) demonstrate the superiority of our proposed framework over the state-of-the-arts.

Problem

• Adapting deep models to mixed distribution shifts in real-world scenarios
• Addressing performance drops under heterogeneous, unlabeled data streams
• Separating domain-specific variations using frequency-domain analysis for robust adaptation

Innovation

• Frequency-domain analysis for domain separation
• Decentralized learning for robust adaptation
• Texture cues to handle mixed shifts