Privacy Amplification for BandMF via $b$-Min-Sep Subsampling

2026-02-10

AI Summary

This work addresses the challenge of privacy amplification in differentially private stochastic gradient descent under band-like correlated noise (BandMF). We propose a novel $b$-min-sep subsampling mechanism that leverages the Markov structure inherent in the subsampling process and integrates it with a dynamic programming–based Monte Carlo privacy accounting method to enable nearly tight privacy analysis. This mechanism unifies and generalizes Poisson and balls-in-bins sampling schemes, offering analytical tractability while achieving superior privacy amplification over cyclic Poisson sampling in the low-to-moderate noise regime for the first time. It also naturally accommodates user-level differential privacy with multi-membership. Experimental results confirm the theoretical advantages: significantly stronger privacy guarantees in low-to-moderate noise settings and comparable performance under high noise.

Technology Category

• Machine Learning: Privacy
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We study privacy amplification for BandMF, i.e., DP-SGD with correlated noise across iterations via a banded correlation matrix. We propose $b$-min-sep subsampling, a new subsampling scheme that generalizes Poisson and balls-in-bins subsampling, extends prior practical batching strategies for BandMF, and enables stronger privacy amplification than cyclic Poisson while preserving the structural properties needed for analysis. We give a near-exact privacy analysis using Monte Carlo accounting, based on a dynamic program that leverages the Markovian structure in the subsampling procedure. We show that $b$-min-sep matches cyclic Poisson subsampling in the high noise regime and achieves strictly better guarantees in the mid-to-low noise regime, with experimental results that bolster our claims. We further show that unlike previous BandMF subsampling schemes, our $b$-min-sep subsampling naturally extends to the multi-attribution user-level privacy setting.

Problem

• Privacy Amplification
• BandMF
• Subsampling
• Differential Privacy
• User-level Privacy

Innovation

• privacy amplification
• b-min-sep subsampling
• BandMF
• user-level privacy
• Monte Carlo accounting