ALIGN-FL: Architecture-independent Learning through Invariant Generative component sharing in Federated Learning

2025-12-15

AI Summary

To address the privacy-performance trade-off in federated learning (FL) involving cross-domain heterogeneous clients under highly non-IID data, this paper proposes a generative component-selective sharing framework: only lightweight VAE generators—not full models—are uploaded, enabling privacy-preserving knowledge aggregation. Methodologically, we innovatively integrate DP-SGD with adaptive gradient clipping and Lipschitz-regularized VAE decoders to jointly ensure differential privacy guarantees and robust reconstruction of anomalous samples. We further design a state-aware client support mechanism and a synthetic-sample-driven global training paradigm on the server. As a model-agnostic solution, our approach achieves a +12.3% accuracy gain on cross-domain anomalous scenarios (MNIST/Fashion-MNIST) and maintains an excellent privacy-utility balance at ε = 2.0, demonstrating strong applicability to high-sensitivity, cross-institutional collaborations—e.g., healthcare and finance.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We present ALIGN-FL, a novel approach to distributed learning that addresses the challenge of learning from highly disjoint data distributions through selective sharing of generative components. Instead of exchanging full model parameters, our framework enables privacy-preserving learning by transferring only generative capabilities across clients, while the server performs global training using synthetic samples. Through complementary privacy mechanisms: DP-SGD with adaptive clipping and Lipschitz regularized VAE decoders and a stateful architecture supporting heterogeneous clients, we experimentally validate our approach on MNIST and Fashion-MNIST datasets with cross-domain outliers. Our analysis demonstrates that both privacy mechanisms effectively map sensitive outliers to typical data points while maintaining utility in extreme Non-IID scenarios typical of cross-silo collaborations. Index Terms: Client-invariant Learning, Federated Learning (FL), Privacy-preserving Generative Models, Non-Independent and Identically Distributed (Non-IID), Heterogeneous Architectures

Problem

• Addresses learning from highly disjoint data distributions in federated learning
• Enables privacy-preserving learning by sharing only generative components
• Maintains utility in extreme Non-IID scenarios with cross-domain outliers

Innovation

• Selectively shares generative components across clients
• Uses DP-SGD and Lipschitz regularized VAE decoders
• Supports heterogeneous clients with a stateful architecture