An Investigation of Memorization Risk in Healthcare Foundation Models

2025-10-14

AI Summary

This study systematically assesses the risk of unintended memorization of patient-sensitive information—and associated privacy threats, particularly for vulnerable populations—in medical foundation models trained on large-scale de-identified electronic health records (EHRs). Method: We propose the first black-box privacy evaluation framework tailored to structured medical data, innovatively disentangling model generalization from harmful memorization along both embedding and generative layers. Our approach combines embedding similarity analysis with customized generative probing techniques to establish a reproducible EHR privacy risk assessment pipeline. Contribution/Results: We validate the framework across multiple publicly available medical foundation models, demonstrating its effectiveness in detecting privacy leakage. To foster community-wide adoption, we open-source the evaluation toolkit, enabling collaborative, standardized privacy risk assessment in healthcare AI. The framework advances rigorous, empirically grounded privacy auditing for clinical language models while highlighting critical vulnerabilities in current de-identification practices.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Foundation models trained on large-scale de-identified electronic health records (EHRs) hold promise for clinical applications. However, their capacity to memorize patient information raises important privacy concerns. In this work, we introduce a suite of black-box evaluation tests to assess privacy-related memorization risks in foundation models trained on structured EHR data. Our framework includes methods for probing memorization at both the embedding and generative levels, and aims to distinguish between model generalization and harmful memorization in clinically relevant settings. We contextualize memorization in terms of its potential to compromise patient privacy, particularly for vulnerable subgroups. We validate our approach on a publicly available EHR foundation model and release an open-source toolkit to facilitate reproducible and collaborative privacy assessments in healthcare AI.

Problem

• Evaluating privacy risks from patient data memorization in healthcare foundation models
• Distinguishing harmful memorization from model generalization in clinical settings
• Assessing memorization vulnerabilities for vulnerable patient subgroups in EHR models

Innovation

• Black-box evaluation tests for memorization risks
• Probing memorization at embedding and generative levels
• Open-source toolkit for reproducible privacy assessments