Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation

2025-07-14

AI Summary

Federated learning (FL) faces three key challenges in resource-constrained edge environments: high annotation cost in target domains, significant covariate shift across clients, and communication- and energy-limited frequent model updates. To address low-shot target-domain adaptation, this paper proposes FedAcross+, a lightweight framework that freezes the pre-trained backbone and classifier while optimizing only a compact, learnable adapter layer. It supports streaming data processing and sporadic model updates, enabling robust adaptation to non-stationary edge environments. Crucially, FedAcross+ operates in a fully unsupervised manner on the target domain—requiring only a few (even single) unlabeled target samples for effective domain adaptation. Extensive experiments demonstrate that FedAcross+ significantly mitigates domain shift on low-resource edge devices, improves generalization, and achieves favorable trade-offs among communication efficiency, computational overhead, and deployment sustainability—establishing a practical new paradigm for edge-aware federated domain adaptation.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Internet of Things, Sensor Networks & Smart Cities
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Federated Learning has emerged as a leading paradigm for decentralized, privacy-preserving learning, particularly relevant in the era of interconnected edge devices equipped with sensors. However, the practical implementation of Federated Learning faces three primary challenges: the need for human involvement in costly data labelling processes for target adaptation, covariate shift in client device data collection due to environmental factors affecting sensors, leading to discrepancies between source and target samples, and the impracticality of continuous or regular model updates in resource-constrained environments due to limited data transmission capabilities and technical constraints on channel availability and energy efficiency. To tackle these issues, we expand upon an efficient and scalable Federated Learning framework tailored for real-world client adaptation in industrial settings. This framework leverages a pre-trained source model comprising a deep backbone, an adaptation module, and a classifier running on a powerful server. By freezing the backbone and classifier during client adaptation on resource-constrained devices, we allow the domain adaptive linear layer to handle target domain adaptation, thus minimizing overall computational overhead. Furthermore, this setup, designated as FedAcross+, is extended to encompass the processing of streaming data, thereby rendering the solution suitable for non-stationary environments. Extensive experimental results demonstrate the effectiveness of FedAcross+ in achieving competitive adaptation on low-end client devices with limited target samples, successfully addressing the challenge of domain shift. Moreover, our framework accommodates sporadic model updates within resource-constrained environments, ensuring practical and seamless deployment.

Problem

• Reduces human involvement in costly data labeling for target adaptation
• Addresses covariate shift in client data due to environmental sensor factors
• Enables efficient model updates in resource-constrained environments

Innovation

• Federated Learning with frozen backbone and classifier
• Domain adaptive linear layer for target adaptation
• FedAcross+ supports streaming data in non-stationary environments