Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data

2025-03-03

AI Summary

This paper addresses the critical legal question of whether large language models (LLMs) constitute “personal data” under data protection laws—such as the GDPR—due to their capacity to memorize and reproduce training data containing personal information. Method: Through interdisciplinary analysis integrating legal doctrine and machine learning, empirical evaluation of model memorization, and systematic GDPR applicability reasoning, the study rigorously examines regulatory obligations across the LLM lifecycle. Contribution/Results: It establishes, for the first time, that an LLM itself may qualify as a “personal data carrier” under the GDPR, thereby triggering data subjects’ rights of access, rectification, and erasure. The paper proposes a comprehensive, lifecycle-aligned compliance framework spanning data collection, model training, deployment, and inference, and clarifies legal liability pathways for personal data leakage from LLMs. These findings foster cross-disciplinary collaboration between AI researchers and legal scholars, providing both theoretical grounding and actionable guidance for legally compliant AI development.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Does GPT know you? The answer depends on your level of public recognition; however, if your information was available on a website, the answer is probably yes. All Large Language Models (LLMs) memorize training data to some extent. If an LLM training corpus includes personal data, it also memorizes personal data. Developing an LLM typically involves processing personal data, which falls directly within the scope of data protection laws. If a person is identified or identifiable, the implications are far-reaching: the AI system is subject to EU General Data Protection Regulation requirements even after the training phase is concluded. To back our arguments: (1.) We reiterate that LLMs output training data at inference time, be it verbatim or in generalized form. (2.) We show that some LLMs can thus be considered personal data on their own. This triggers a cascade of data protection implications such as data subject rights, including rights to access, rectification, or erasure. These rights extend to the information embedded with-in the AI model. (3.) This paper argues that machine learning researchers must acknowledge the legal implications of LLMs as personal data throughout the full ML development lifecycle, from data collection and curation to model provision on, e.g., GitHub or Hugging Face. (4.) We propose different ways for the ML research community to deal with these legal implications. Our paper serves as a starting point for improving the alignment between data protection law and the technical capabilities of LLMs. Our findings underscore the need for more interaction between the legal domain and the ML community.

Problem

• LLMs memorize personal data from training corpora.
• LLMs may be considered personal data under GDPR.
• Legal implications of LLMs must be addressed in ML lifecycle.

Innovation

• LLMs memorize personal data from training
• LLMs subject to GDPR post-training
• Propose legal compliance strategies for ML