SoK: Challenges in Tabular Membership Inference Attacks

2026-01-22

AI Summary

This study addresses the overestimation of membership inference attack (MIA) effectiveness, insufficient evaluation practices, and the overlooked threat posed by external adversaries in tabular data settings. Through a systematic investigation of MIA performance and limitations under both centralized and federated learning paradigms, the work establishes a unified taxonomy to comprehensively evaluate diverse attack strategies, defense mechanisms, and surrogate model substitution approaches. The findings reveal, for the first time, that the overall efficacy of MIAs on tabular data is substantially lower than previously assumed, although individual high-risk instances remain vulnerable. Furthermore, the study demonstrates that surrogate models with architectures differing from the target can significantly enhance attack success rates and underscores the tangible privacy risks introduced by external attackers in federated learning environments.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Membership Inference Attacks (MIAs) are currently a dominant approach for evaluating privacy in machine learning applications. Despite their significance in identifying records belonging to the training dataset, several concerns remain unexplored, particularly with regard to tabular data. In this paper, first, we provide an extensive review and analysis of MIAs considering two main learning paradigms: centralized and federated learning. We extend and refine the taxonomy for both. Second, we demonstrate the efficacy of MIAs in tabular data using several attack strategies, also including defenses. Furthermore, in a federated learning scenario, we consider the threat posed by an outsider adversary, which is often neglected. Third, we demonstrate the high vulnerability of single-outs (records with a unique signature) to MIAs. Lastly, we explore how MIAs transfer across model architectures. Our results point towards a general poor performance of these attacks in tabular data which contrasts with previous state-of-the-art. Notably, even attacks with limited attack performance can still successfully expose a large portion of single-outs. Moreover, our findings suggest that using different surrogate models makes MIAs more effective.

Problem

• Membership Inference Attacks
• Tabular Data
• Privacy
• Federated Learning
• Single-outs

Innovation

• Membership Inference Attacks
• Tabular Data
• Federated Learning
• Single-outs
• Surrogate Models