Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs

2026-08-13

AI Summary

This work addresses a critical privacy vulnerability in multimodal large language models for document understanding: when visual evidence is insufficient, these models often rely on memorized field correlations from training data to infer missing content, leading to relational leakage of sensitive personal information. To mitigate this risk, the paper introduces the first systematic analysis of such privacy leakage mechanisms and proposes a Dynamic Relation Unlearning Framework (DRUF), which integrates a relation decoupling unlearning module with a dynamic set updating mechanism to suppress high-risk field associations while preserving essential information extraction performance. Additionally, the authors construct DocPrivacyBench, the first privacy evaluation benchmark tailored to scenarios with missing visual evidence. Experiments demonstrate that DRUF significantly reduces privacy leakage risk—outperforming the strongest baseline by 4.8 percentage points—without compromising robustness in information extraction.

Abstract

While the privacy risks of multimodal large language models (MLLMs) have drawn significant attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on document understanding MLLMs for identity document processing, this paper investigates the privacy issues inherent in Key Information Extraction (KIE) tasks. We reveal that when input images lack sufficient visual evidence, these models often rely on memorized field relations from training data to infer missing content, thereby leaking multiple correlated fields containing sensitive personal information. To mitigate this risk, we make three key contributions.First, we propose the Dynamic Relational Unlearning Framework (DRUF) which comprises a Relational Decoupling Unlearning (RDU) module and a dynamic set update mechanism. It suppresses the leakage of high-risk field pairs while preserving KIE performance.Second, we introduce DocPrivacyBench, a novel benchmark to systematically evaluate a model's susceptibility to privacy leakage under conditions of absent or minimal visual evidence.Third, we evaluate three MLLMs and six unlearning methods using this benchmark, assessing both post-unlearning leakage suppression and utility preservation.Our results demonstrate that existing MLLMs consistently exhibit privacy leakage when visual evidence is scarce, particularly on noisier datasets. In contrast, DRUF outperforms the strongest baseline by improving leakage suppression by 4.8 percentage points, effectively mitigating privacy risks while maintaining robust document information extraction performance.

Problem

• relational privacy leakage
• document MLLMs
• Key Information Extraction
• visual evidence
• privacy risk

Innovation

• Relational Privacy Leakage
• Dynamic Relational Unlearning Framework
• Key Information Extraction
• DocPrivacyBench
• Multimodal Large Language Models