On the Robustness of LDP Protocols for Numerical Attributes under Data Poisoning Attacks

2024-03-28

arXiv.org

AI Summary

This paper addresses the lack of robustness in Local Differential Privacy (LDP) protocols for numerical attributes under malicious data poisoning attacks. It systematically evaluates the resilience of two prominent protocol families—Counting Frequency Oracle (CFO) and distribution-reconstruction methods (e.g., Square Wave)—against such attacks. To quantify cross-protocol vulnerability, the authors introduce Cross-Protocol Attack Gain, a novel metric revealing that hash domain size affects robustness far more than utility. They further propose a zero-shot attack detection mechanism grounded in reconstructed distribution statistics, enabling server-side identification of malicious clients without prior knowledge of attack patterns. Experiments demonstrate that server-deployed Square Wave and CFO significantly outperform client-side CFO in robustness. The proposed detector achieves substantial accuracy improvements under challenging conditions—including high noise and low sampling rates—providing both theoretical foundations and practical tools for reliable LDP deployment in untrusted environments.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Recent studies reveal that local differential privacy (LDP) protocols are vulnerable to data poisoning attacks where an attacker can manipulate the final estimate on the server by leveraging the characteristics of LDP and sending carefully crafted data from a small fraction of controlled local clients. This vulnerability raises concerns regarding the robustness and reliability of LDP in hostile environments. In this paper, we conduct a systematic investigation of the robustness of state-of-the-art LDP protocols for numerical attributes, i.e., categorical frequency oracles (CFOs) with binning and consistency, and distribution reconstruction. We evaluate protocol robustness through an attack-driven approach and propose new metrics for cross-protocol attack gain measurement. The results indicate that Square Wave and CFO-based protocols in the Server setting are more robust against the attack compared to the CFO-based protocols in the User setting. Our evaluation also unfolds new relationships between LDP security and its inherent design choices. We found that the hash domain size in local-hashing-based LDP has a profound impact on protocol robustness beyond the well-known effect on utility. Further, we propose a zero-shot attack detection by leveraging the rich reconstructed distribution information. The experiment show that our detection significantly improves the existing methods and effectively identifies data manipulation in challenging scenarios.

Problem

• LDP Robustness
• Malicious Data Corruption
• Algorithm Reliability

Innovation

• LDP Robustness
• Unsupervised Anomaly Detection
• Square Wave and CFOs Algorithms