A LINDDUN-based Privacy Threat Modeling Framework for GenAI

2026-03-06

AI Summary

This work proposes a privacy threat modeling framework tailored for generative artificial intelligence (GenAI) systems, addressing the significant privacy challenges they pose when handling sensitive data. Traditional threat modeling approaches fall short in capturing the unique risks inherent to GenAI, prompting the extension of the classic LINDDUN methodology to encompass three core threat categories and introduce 100 GenAI-specific privacy threat examples. The framework integrates a systematic literature review with an in-depth case study of a conversational AI agent, demonstrating its effectiveness in enabling systematic, fine-grained identification and assessment of privacy risks in GenAI applications. This represents the first structured approach to modeling privacy threats in GenAI, thereby filling a critical gap in the current landscape of privacy engineering for artificial intelligence systems.

Technology Category

• Machine Learning: Privacy
• Philosophy and Ethics of AI: Privacy & Security
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Security and privacy of machine learning and AI applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

As generative AI (GenAI) systems become increasingly prevalent across various technological stacks, the question of how such systems handle sensitive and personal data flows becomes increasingly important. Specifically, both the ability to harness and process large swaths of information as well as their stochastic nature raise key concerns related to both security and privacy. Unfortunately, while some of the traditional security threat modeling can effectively identify certain violations, privacy-related issues are often overlooked. To respond to these challenges, we introduce a novel domain-specific privacy threat modeling framework to support the privacy threat analysis of GenAI-based applications. This framework is constructed through a two-pronged approach: (1) a systematic review of the emerging literature on GenAI privacy threats, and (2) a case-driven application to a representative Chatbot system. These efforts yield a foundational GenAI privacy threat modeling framework built on LINDDUN. The new framework affects three out of the seven privacy threat types of LINDDUN and introduces 100 new GenAI examples to the knowledge base. Its effectiveness is validated on an AI Agent system, which demonstrates that a comprehensive privacy analysis can be supported by the new framework.

Problem

• Generative AI
• Privacy Threats
• Threat Modeling
• Data Privacy
• LINDDUN

Innovation

• LINDDUN
• Generative AI
• Privacy Threat Modeling
• Domain-specific Framework
• Chatbot