FANS: Federated Adaptive Network Search Learning for Heterogeneous Devices

2026-09-05

AI Summary

为解决异构设备联邦学习中模型配置受限问题，提出FANS框架及FPS算法，通过共享架构空间和并行训练优化模型性能。

Abstract

Heterogeneous Federated Learning (HFL) aims to train models across devices with diverse resource budgets while preserving data privacy. Existing HFL methods typically bind training to a small predefined menu of model configurations, which limits architectural coverage. To address this bottleneck, we introduce Federated Adaptive Network Search (FANS), a hypernetwork-based framework that learns a shared architecture space rather than a fixed set of client models. To optimize this shared space efficiently, we propose the Federated Parallel Scaling (FPS) algorithm, which jointly trains multiple sampled subnetworks in parallel with self-distillation so that larger sampled subnetworks can supervise smaller ones during local updates. We evaluate FANS on CIFAR-10, CIFAR-100, and MNLI using ResNet-18, DenseNet-121, and BERT-base, respectively. Across all benchmarks, FANS expands the feasible subnetwork pool by orders of magnitude (e.g., 4,680 candidates for ResNet-18 vs. 4 in existing methods) and improves the average accuracy-efficiency trade-off relative to representative HFL baselines. Device heterogeneity is emulated through resource tiers, and evaluation covers accuracy, parameter count, and MACs.

Problem

• Federated Learning
• Heterogeneous Devices
• Model Configurations
• Architecture Space
• Resource Budgets

Innovation

• Federated Adaptive Network Search
• Hypernetwork-based Framework
• Federated Parallel Scaling
• Self-Distillation