DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacy

2026-04-17

AI Summary

This work addresses the challenge of automating the design and verification of differentially private (DP) algorithms, which currently rely heavily on expert knowledge. To this end, the authors introduce DPrivBench, the first systematic benchmark for evaluating large language models’ (LLMs’) reasoning capabilities in DP. The benchmark encompasses diverse topics and difficulty levels while deliberately avoiding shortcut solutions based on pattern matching. Empirical evaluation using DPrivBench reveals that although current LLMs can handle classical DP mechanisms, they exhibit significant gaps when reasoning about more complex DP algorithms. By integrating automated reasoning, DP theory, and adversarial benchmark design, this study establishes a foundational framework for assessing and advancing automated DP verification.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Differential privacy (DP) has a wide range of applications for protecting data privacy, but designing and verifying DP algorithms requires expert-level reasoning, creating a high barrier for non-expert practitioners. Prior works either rely on specialized verification languages that demand substantial domain expertise or remain semi-automated and require human-in-the-loop guidance. In this work, we investigate whether large language models (LLMs) can automate DP reasoning. We introduce DPrivBench, a benchmark in which each instance asks whether a function or algorithm satisfies a stated DP guarantee under specified assumptions. The benchmark is carefully designed to cover a broad range of DP topics, span diverse difficulty levels, and resist shortcut reasoning through trivial pattern matching. Experiments show that while the strongest models handle textbook mechanisms well, all models struggle with advanced algorithms, revealing substantial gaps in current DP reasoning capabilities. Through further analytic study and failure-mode analysis, we identify several promising directions for improving automated DP reasoning. Our benchmark provides a solid foundation for developing and evaluating such methods, and complements existing benchmarks for mathematical reasoning.

Problem

• differential privacy
• automated reasoning
• large language models
• algorithm verification
• privacy-preserving

Innovation

• Differential Privacy
• Large Language Models
• Automated Reasoning
• Benchmarking
• Privacy Verification