Understanding Privacy Risks in Code Models Through Training Dynamics: A Causal Approach

2025-12-08

AI Summary

Large language models for code (LLM4Code) trained on open-source repositories containing personally identifiable information (PII) pose significant privacy leakage risks; however, prior work overlooks the heterogeneity across PII types. Method: We propose the first type-aware structural causal model to quantify the causal relationship between PII learnability and leakage risk from a training dynamics perspective. Leveraging a multi-type PII dataset, fine-tuning across model scales, and empirical analysis of real training trajectories, we systematically measure how PII types—including IP addresses, cryptographic keys, and fuzzy identifiers—differ in learnability and leakage propensity. Results: We find that easily learnable PII (e.g., IPs) exhibits significantly higher leakage frequency, while hard-to-learn types (e.g., keys) leak less frequently; fuzzy identifiers show divergent behavior. Based on these findings, we introduce a learnability-aware defense framework, providing both theoretical grounding and practical guidance for fine-grained privacy risk assessment and mitigation in LLM4Code.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language models for code (LLM4Code) have greatly improved developer productivity but also raise privacy concerns due to their reliance on open-source repositories containing abundant personally identifiable information (PII). Prior work shows that commercial models can reproduce sensitive PII, yet existing studies largely treat PII as a single category and overlook the heterogeneous risks among different types. We investigate whether distinct PII types vary in their likelihood of being learned and leaked by LLM4Code, and whether this relationship is causal. Our methodology includes building a dataset with diverse PII types, fine-tuning representative models of different scales, computing training dynamics on real PII data, and formulating a structural causal model to estimate the causal effect of learnability on leakage. Results show that leakage risks differ substantially across PII types and correlate with their training dynamics: easy-to-learn instances such as IP addresses exhibit higher leakage, while harder types such as keys and passwords leak less frequently. Ambiguous types show mixed behaviors. This work provides the first causal evidence that leakage risks are type-dependent and offers guidance for developing type-aware and learnability-aware defenses for LLM4Code.

Problem

• Investigates varying privacy risks among different PII types in code models
• Assesses causal relationship between learnability and leakage of sensitive information
• Provides evidence for type-dependent defenses in large language models for code

Innovation

• Causal analysis of PII leakage risks in code models
• Training dynamics to measure learnability of different PII types
• Structural causal model linking learnability to leakage outcomes