Privacy Preserving Gossip Learning

2026-09-13

AI Summary

提出了一种去中心化的隐私保护学习算法，通过每个代理持有单个私有样本和共享模型，并结合Tuning without Forgetting方法及差分隐私技术来保护数据隐私。

Abstract

We propose a decentralized privacy-preserving learning algorithm in which each agent holds a single private sample and a shared model. Samples are learned sequentially, and each update must preserve the endpoint mappings at previously learned samples while protecting private data. This gives each agent three roles: (i) a learner that updates the model parameters, (ii) a teacher whose sample is learned at the current iteration, and (iii) a protected agent whose sample has already been learned. We build on Tuning without Forgetting (TwF) method to preserve previously learned mappings and show that TwF provides an indistinguishability guarantee for the learner whenever the set of protected agents contains another sample with the same label. For the teacher, we formulate a minimax optimal control problem that models the differential privacy noise as a worst-case disturbance to prevent performance loss while maintaining the same level of privacy for the gradient. For the protected agents, we compute the projections locally and aggregate them using a private push-sum gossip protocol. We prove geometric convergence of the decentralized gossip algorithm and of the distributed projection for TwF.

Problem

• Decentralized Learning
• Privacy Preservation
• Gossip Learning

Innovation

• decentralized privacy-preserving learning
• Tuning without Forgetting (TwF)
• differential privacy
• minimax optimal control
• private push-sum gossip protocol