MIDST Challenge at SaTML 2025: Membership Inference over Diffusion-models-based Synthetic Tabular data

2026-03-19

AI Summary

This study systematically evaluates the privacy risks of synthetic tabular data generated by diffusion models under membership inference attacks. It introduces the first tailored black-box and white-box attack methodologies specifically designed for both single-table and multi-relational tabular data, accommodating mixed data types and referential integrity constraints. By doing so, the work addresses a critical gap in privacy assessment for diffusion models applied to complex structured data. Through comprehensive privacy quantification experiments, this research not only advances the development of novel membership inference attacks tailored to structured data but also establishes the first empirical benchmark for evaluating the privacy-preserving capabilities of diffusion models in tabular data synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Privacy
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Synthetic data is often perceived as a silver-bullet solution to data anonymization and privacy-preserving data publishing. Drawn from generative models like diffusion models, synthetic data is expected to preserve the statistical properties of the original dataset while remaining resilient to privacy attacks. Recent developments of diffusion models have been effective on a wide range of data types, but their privacy resilience, particularly for tabular formats, remains largely unexplored. MIDST challenge sought a quantitative evaluation of the privacy gain of synthetic tabular data generated by diffusion models, with a specific focus on its resistance to membership inference attacks (MIAs). Given the heterogeneity and complexity of tabular data, multiple target models were explored for MIAs, including diffusion models for single tables of mixed data types and multi-relational tables with interconnected constraints. MIDST inspired the development of novel black-box and white-box MIAs tailored to these target diffusion models as a key outcome, enabling a comprehensive evaluation of their privacy efficacy. The MIDST GitHub repository is available at https://github.com/VectorInstitute/MIDST

Problem

• membership inference
• diffusion models
• synthetic tabular data
• privacy
• data anonymization

Innovation

• diffusion models
• synthetic tabular data
• membership inference attacks
• privacy evaluation
• black-box MIA