Winning the MIDST Challenge: New Membership Inference Attacks on Diffusion Models for Tabular Data Synthesis

2025-03-15

AI Summary

Existing diffusion-based synthetic tabular data generators lack rigorous privacy evaluation; mainstream approaches rely solely on heuristic metrics or weak membership inference attacks (MIAs), substantially underestimating real-world privacy risks. Method: We systematically demonstrate that state-of-the-art MIAs—designed for images—consistently fail in tabular settings, identifying noise initialization as a critical factor undermining attack efficacy. To address this, we propose the first machine learning–driven MIA framework tailored to diffusion-based tabular synthesis, leveraging multi-noise-level and time-step–dependent loss features. Our approach replaces manual hyperparameter tuning with a lightweight MLP that automatically learns discriminative membership signals. Contribution/Results: Our method achieved first place across all tracks of the SaTML 2025 MIDST Challenge, significantly outperforming baselines and providing compelling empirical evidence of substantial privacy leakage in diffusion-generated tabular data.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Diffusion Models for Vision

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Tabular data synthesis using diffusion models has gained significant attention for its potential to balance data utility and privacy. However, existing privacy evaluations often rely on heuristic metrics or weak membership inference attacks (MIA), leaving privacy risks inadequately assessed. In this work, we conduct a rigorous MIA study on diffusion-based tabular synthesis, revealing that state-of-the-art attacks designed for image models fail in this setting. We identify noise initialization as a key factor influencing attack efficacy and propose a machine-learning-driven approach that leverages loss features across different noises and time steps. Our method, implemented with a lightweight MLP, effectively learns membership signals, eliminating the need for manual optimization. Experimental results from the MIDST Challenge @ SaTML 2025 demonstrate the effectiveness of our approach, securing first place across all tracks. Code is available at https://github.com/Nicholas0228/Tartan_Federer_MIDST.

Problem

• Assessing privacy risks in diffusion-based tabular data synthesis
• Improving membership inference attacks for tabular data models
• Developing a machine-learning-driven approach for privacy evaluation

Innovation

• Machine-learning-driven MIA for tabular data
• Leverages loss features across noises and steps
• Lightweight MLP eliminates manual optimization