FedHypeVAE: Federated Learning with Hypernetwork Generated Conditional VAEs for Differentially Private Embedding Sharing

2026-01-02

arXiv.org

AI Summary

This work proposes FedHypeVAE, a novel framework addressing the challenges of poor embedding-level data generation quality and heightened privacy risks from gradient leakage in federated learning with non-IID clients. FedHypeVAE leverages a shared hypernetwork to generate personalized conditional variational autoencoder (CVAE) decoders and class-conditional priors for each client, enabling privacy-preserving data synthesis in the embedding space. The approach innovatively integrates hypernetworks, differential privacy via gradient clipping and noise injection, local maximum mean discrepancy (MMD) for distribution alignment, and Lipschitz regularization to jointly achieve generator personalization, rigorous privacy guarantees, and cross-client distribution alignment. Experimental results demonstrate that FedHypeVAE significantly improves synthetic embedding quality and model stability under non-IID settings while supporting controllable multi-domain data generation.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Privacy-enhancing technologies

Abstract

Federated data sharing promises utility without centralizing raw data, yet existing embedding-level generators struggle under non-IID client heterogeneity and provide limited formal protection against gradient leakage. We propose FedHypeVAE, a differentially private, hypernetwork-driven framework for synthesizing embedding-level data across decentralized clients. Building on a conditional VAE backbone, we replace the single global decoder and fixed latent prior with client-aware decoders and class-conditional priors generated by a shared hypernetwork from private, trainable client codes. This bi-level design personalizes the generative layerrather than the downstream modelwhile decoupling local data from communicated parameters. The shared hypernetwork is optimized under differential privacy, ensuring that only noise-perturbed, clipped gradients are aggregated across clients. A local MMD alignment between real and synthetic embeddings and a Lipschitz regularizer on hypernetwork outputs further enhance stability and distributional coherence under non-IID conditions. After training, a neutral meta-code enables domain agnostic synthesis, while mixtures of meta-codes provide controllable multi-domain coverage. FedHypeVAE unifies personalization, privacy, and distribution alignment at the generator level, establishing a principled foundation for privacy-preserving data synthesis in federated settings. Code: github.com/sunnyinAI/FedHypeVAE

Problem

• federated learning
• differential privacy
• embedding sharing
• non-IID
• gradient leakage

Innovation

• Federated Learning
• Hypernetwork
• Conditional VAE
• Differential Privacy
• Non-IID