Distributed Deep Variational Approach for Privacy-preserving Data Release

2026-05-04

AI Summary

This work addresses the challenge of sensitive attribute leakage through model updates in federated learning, which undermines high privacy guarantees. The authors propose the Gaussian Privacy Protector (GPP), the first framework to integrate a variational privacy mechanism into federated learning. GPP achieves end-to-end instance-level privacy without transmitting sensitive labels by minimizing a variational lower bound on the mutual information between published representations and sensitive attributes. The method employs a deep variational encoder with a Lagrangian multiplier–based multi-objective loss function, enabling a tunable trade-off between utility and privacy. Experiments on MNIST, CelebA, and HAPT datasets demonstrate that GPP incurs only about a 1% utility drop compared to an unconstrained autoencoder while reducing the AUC of attribute inference attacks to near-random levels.

Abstract

Federated learning (FL) lets distributed nodes train a shared model without exchanging their raw data, but in privacy-sensitive deployments medical sensors, IoT devices, wearables the protection offered by keeping data local is incomplete: gradients, model updates, and the released representations themselves can leak sensitive attributes. We propose the \emph{Gaussian Privacy Protector} (GPP), a data-release framework for continuous, high-dimensional inputs that learns a stochastic encoder mapping raw data to a low-dimensional sanitized representation. The encoder is trained against a variational lower bound on the mutual information between the released representation and a designated sensitive attribute, while a separate cross-entropy term preserves a designated utility attribute, with a Lagrange multiplier $β$ controlling the trade-off. We then extend GPP to the federated setting, in which each client trains a local encoder, sensitive labels never leave the client, and the aggregator receives only sanitized representations giving instance-level privacy protection in addition to the standard ``raw data stays local'' guarantee of FL. We evaluate GPP on MNIST (digit-sum utility, parity sensitive), CelebA (smiling vs.\ gender), and HAPT-Recognition (activity vs.\ subject identity). Across all three benchmarks, GPP attains utility within roughly one percentage point of an unconstrained autoencoder baseline while reducing the adversary's AUC to near random guessing.

Problem

• privacy-preserving data release
• federated learning
• sensitive attribute leakage
• utility-privacy trade-off
• distributed learning

Innovation

• privacy-preserving representation
• variational mutual information
• federated learning
• stochastic encoder
• utility-privacy trade-off