Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Masking

2025-09-18

AI Summary

Split Federated Learning (SFL) faces three key challenges: intermediate activations are vulnerable to data reconstruction attacks; noise-based defenses degrade model accuracy; and data/system heterogeneity exacerbates performance degradation. To address these, we propose PM-SFL—a privacy- and heterogeneity-aware SFL framework. Its core contributions are: (1) probabilistic masking training, which replaces explicit noise injection with structured stochasticity to achieve high-fidelity privacy protection; (2) personalized mask learning, dynamically adapting masks to client-specific data distributions; and (3) adaptive model partitioning coupled with layer-wise knowledge compensation to mitigate system heterogeneity. Theoretical analysis and extensive experiments across image classification and wireless sensing tasks demonstrate that PM-SFL simultaneously ensures strong privacy guarantees and significantly improves accuracy (+3.2–5.8%) and communication efficiency (37% reduction in upload volume), with exceptional robustness under high heterogeneity.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Privacy-enhancing technologies
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Split Federated Learning (SFL) has emerged as an efficient alternative to traditional Federated Learning (FL) by reducing client-side computation through model partitioning. However, exchanging of intermediate activations and model updates introduces significant privacy risks, especially from data reconstruction attacks that recover original inputs from intermediate representations. Existing defenses using noise injection often degrade model performance. To overcome these challenges, we present PM-SFL, a scalable and privacy-preserving SFL framework that incorporates Probabilistic Mask training to add structured randomness without relying on explicit noise. This mitigates data reconstruction risks while maintaining model utility. To address data heterogeneity, PM-SFL employs personalized mask learning that tailors submodel structures to each client's local data. For system heterogeneity, we introduce a layer-wise knowledge compensation mechanism, enabling clients with varying resources to participate effectively under adaptive model splitting. Theoretical analysis confirms its privacy protection, and experiments on image and wireless sensing tasks demonstrate that PM-SFL consistently improves accuracy, communication efficiency, and robustness to privacy attacks, with particularly strong performance under data and system heterogeneity.

Problem

• Addressing privacy risks from intermediate data exchange in Split Federated Learning
• Mitigating performance degradation from noise-based privacy defenses
• Handling data and system heterogeneity across client devices

Innovation

• Probabilistic Mask training for privacy
• Personalized mask learning for data heterogeneity
• Layer-wise knowledge compensation for system heterogeneity