PRISM: Privacy-Preserving Improved Stochastic Masking for Federated Generative Models

2025-03-11

AI Summary

To address instability in generative model training, high communication overhead, privacy risks, and model redundancy in non-IID federated learning, this paper proposes PRISM—a novel framework for federated generative modeling. PRISM introduces, for the first time, a stochastic binary masking mechanism to efficiently identify sparse “strong lottery-ticket” subnetworks. It incorporates an MMD-based loss to enhance cross-client distribution alignment and devises Mask-Aware Dynamic Averaging (MADA) aggregation for stable, adaptive model fusion. Crucially, PRISM supports native sparsification without post-hoc pruning. Evaluated on MNIST, FMNIST, CelebA, and CIFAR-10, PRISM reduces communication volume by over 60% compared to SOTA methods, compresses model size to less than 5%, satisfies differential privacy guarantees, and—uniquely under combined non-IID and privacy constraints—achieves high-fidelity image generation.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Despite recent advancements in federated learning (FL), the integration of generative models into FL has been limited due to challenges such as high communication costs and unstable training in heterogeneous data environments. To address these issues, we propose PRISM, a FL framework tailored for generative models that ensures (i) stable performance in heterogeneous data distributions and (ii) resource efficiency in terms of communication cost and final model size. The key of our method is to search for an optimal stochastic binary mask for a random network rather than updating the model weights, identifying a sparse subnetwork with high generative performance; i.e., a ``strong lottery ticket''. By communicating binary masks in a stochastic manner, PRISM minimizes communication overhead. This approach, combined with the utilization of maximum mean discrepancy (MMD) loss and a mask-aware dynamic moving average aggregation method (MADA) on the server side, facilitates stable and strong generative capabilities by mitigating local divergence in FL scenarios. Moreover, thanks to its sparsifying characteristic, PRISM yields a lightweight model without extra pruning or quantization, making it ideal for environments such as edge devices. Experiments on MNIST, FMNIST, CelebA, and CIFAR10 demonstrate that PRISM outperforms existing methods, while maintaining privacy with minimal communication costs. PRISM is the first to successfully generate images under challenging non-IID and privacy-preserving FL environments on complex datasets, where previous methods have struggled.

Problem

• Enhances federated generative models' stability in heterogeneous data.
• Reduces communication costs and model size in federated learning.
• Ensures privacy and efficiency in non-IID data environments.

Innovation

• Optimizes stochastic binary masks for efficiency
• Uses MMD loss and MADA for stability
• Generates lightweight models without extra pruning