CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment

2026-08-10

AI Summary

Existing approaches struggle to model how individuals dynamically adjust their privacy disclosure boundaries beyond generic norms in specific contexts, leading to misalignment between large language models and user preferences. To address this, this work introduces the CIDER dataset, comprising 1,650 fine-grained disclosure decisions from 169 users across 60 interpersonal scenarios—each decision encompassing nine information variants—thereby enabling the first systematic capture of real-world privacy boundaries under contextual variation. The study formulates a contextualized privacy preference prediction task grounded in users’ historical behavior. Through large-scale human annotation, context-aware personalized prompt learning, and a cross-model evaluation framework, experiments demonstrate that as few as six historical samples can boost prediction accuracy by up to 11.41 percentage points across twelve models. Notably, large models better leverage semantic context, with Claude Sonnet 4.6 uniquely reducing both false positives and false negatives.

Abstract

Aligning large language models (LLMs) with human privacy preferences requires capturing individuals' disclosure boundaries beyond general privacy norms. However, a gap remains in eliciting such nuanced preferences to evaluate alignment in realistic settings. We introduce CIDER, a dataset of 14,850 human annotations from 169 users, forming 1,650 contextual disclosure boundary sets across 60 interpersonal communication scenarios involving information sharing that violates privacy norms. Each boundary represents a real user's disclosure decisions over 9 sharing variants in a scenario, for a given communication role and AI-mediated condition. We formulate a task in which models predict a user's disclosure decision from historical boundaries, with varying levels of contextual information. Across 12 open and proprietary models, in-context personalization improves prediction accuracy by up to 11.41 percentage points using only 6 historical examples. Larger models such as GPT-5.4 (with medium reasoning effort) and Claude Sonnet 4.6 are better at leveraging semantic context to understand user-specific, context-dependent disclosure preferences for more accurate predictions, while smaller models tend to rely on structured heuristics based on disclosure granularity and identifiability. Personalization generally improves prediction accuracy, but the improvement is often accompanied by imbalanced shifts in false-positive and false-negative rates across models, with only Claude Sonnet 4.6 achieving balanced improvements in both. Our findings reveal both the promise and limitations of inference-time personalization for privacy preference modeling and position CIDER as a resource for advancing personalized privacy alignment.

Problem

• privacy preference alignment
• contextual disclosure boundaries
• large language models
• personalized privacy
• human-AI interaction

Innovation

• contextual disclosure boundaries
• privacy preference alignment
• in-context personalization
• LLM privacy modeling
• CIDER dataset