🤖 AI Summary
In heterogeneous federated learning (FL), varying client model complexities exacerbate membership inference attack (MIA) risks, compromising participant privacy.
Method: We propose a model-agnostic FL framework centered on dynamic random submodel selection: during aggregation, the server stochastically samples substructures from lightweight client models and integrates them into its own large model.
Contribution/Results: We establish the first taxonomy of model-agnostic FL methods and theoretically and empirically demonstrate that randomized submodel selection jointly enhances privacy and utility. Experiments on CIFAR-10 and CIFAR-100 show that our approach reduces MIA success rates by up to 37% while incurring ≤1.2% accuracy loss on the server model. This work bridges a critical gap in multi-strategy, privacy-enhancing FL research—particularly where privacy and utility are co-optimized through structural randomness rather than perturbation-based defenses.
📝 Abstract
Federated Learning (FL) has been proposed as a privacy-preserving solution for machine learning. However, recent works have reported that FL can leak private client data through membership inference attacks. In this paper, we show that the effectiveness of these attacks on the clients negatively correlates with the size of the client's datasets and model complexity. Based on this finding, we study the capabilities of model-agnostic Federated Learning to preserve privacy, as it enables the use of models of varying complexity in the clients. To systematically study this topic, we first propose a taxonomy of model-agnostic FL methods according to the strategies adopted by the clients to select the sub-models from the server's model. This taxonomy provides a framework for existing model-agnostic FL approaches and leads to the proposal of new FL methods to fill the gaps in the taxonomy. Next, we analyze the privacy-performance trade-off of all the model-agnostic FL architectures as per the proposed taxonomy when subjected to 3 different membership inference attacks on the CIFAR-10 and CIFAR-100 vision datasets. In our experiments, we find that randomness in the strategy used to select the server's sub-model to train the clients' models can control the clients' privacy while keeping competitive performance on the server's side.
