PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Data

2026-09-08

AI Summary

为解决个性化学习系统中隐私保护与数据可用性问题，提出PEARL框架综合评估差分隐私合成教育数据的有效性、隐私保护、预测有用性和任务适用性。

Abstract

Personalized learning systems rely on real learner data, including performance, behavior, and demographic information, but these data are highly privacy-sensitive. Differentially private (DP) synthetic data can support system development and educational research while reducing exposure of individual learners. Existing evaluations, however, assess privacy and predictive usefulness separately, without determining whether synthetic learner data remain usable for the intended personalized learning task. We introduce PEARL (Privacy-Equivalence Audit and Release Ledger), which approves a DP synthetic educational dataset only when it passes all required checks of validity, privacy protection, predictive usefulness, and suitability for the intended educational task, while recording why each rejected dataset fails. Across 96 study settings, each defined by a dataset, data-generation method, privacy budget, and random seed, only 12 produced synthetic datasets that passed all applicable PEARL checks. Many privacy-protected datasets were rejected for omitting important outcome groups, such as withdrawn students, or for failing to preserve the order of learning activities. Fairness analysis further showed that some datasets passing privacy and predictive-usefulness checks still yielded unequal at-risk prediction performance across groups defined by disability and socioeconomic background. Moreover, Deep Knowledge Tracing and Self-Attentive Knowledge Tracing learned no meaningful next-response patterns from any tested synthetic knowledge-tracing dataset, showing that privacy protection alone does not guarantee usefulness for dropout prediction, knowledge tracing, or adaptive tutoring.

Problem

• Differentially Private
• Synthetic Data
• Personalized Learning
• Privacy Protection
• Predictive Usefulness

Innovation

• Differentially Private Synthetic Data
• Task-Aware Evaluation
• Privacy-Equivalence Audit and Release Ledger (PEARL)
• Personalized Learning Systems