DPCheatSheet: Using Worked and Erroneous LLM-usage Examples to Scaffold Differential Privacy Implementation

2025-09-15

AI Summary

Programmers lacking expertise in differential privacy (DP) struggle to correctly implement DP algorithms using large language models (LLMs), primarily due to ambiguous prompting and insufficient capability to verify DP compliance of generated code. Method: This paper proposes an LLM-assisted pedagogical approach that synergistically integrates worked examples and erroneous examples—novelly combining these two instructional paradigms for DP programming education. It introduces a structured prompt template and verification guide grounded in expert workflows and empirically observed common errors. Contribution/Results: Experiments demonstrate significant improvements: a 37.2% increase in novices’ ability to detect DP implementation flaws and a 41.5% rise in successful independent generation of DP-compliant code. The method effectively bridges the gap between expert DP practice and novice proficiency.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper explores how programmers without specialized expertise in differential privacy (DP) (i.e., novices) can leverage LLMs to implement DP programs with minimal training. We first conducted a need-finding study with 6 novices and 3 experts to understand how they utilize LLMs in DP implementation. While DP experts can implement correct DP analyses through a few prompts, novices struggle to articulate their requirements in prompts and lack the skills to verify the correctness of the generated code. We then developed DPCheatSheet, an instructional tool that helps novices implement DP using LLMs. DPCheatSheet combines two learning concepts: it annotates an expert's workflow with LLMs as a worked example to bridge the expert mindset to novices, and it presents five common mistakes in LLM-based DP code generation as erroneous examples to support error-driven learning. We demonstrated the effectiveness of DPCheatSheet with an error identification study and an open-ended DP implementation study.

Problem

• Helping novices implement differential privacy using LLMs
• Bridging expertise gap between DP experts and novices
• Addressing error identification in LLM-generated DP code

Innovation

• Instructional tool scaffolds DP implementation
• Worked examples bridge expert mindset to novices
• Error-driven learning with common mistake examples