When Privacy Meets Recovery: The Overlooked Half of Surrogate-Driven Privacy Preservation for MLLM Editing

2025-12-07

AI Summary

Multimodal large language models (MLLMs) face a fundamental trade-off between privacy protection and data recoverability. This work is the first to systematically expose the genuine privacy recovery risks inherent in proxy-data-driven privacy protection. Method: We propose a novel high-fidelity privacy reconstruction paradigm based on editable proxy data. To rigorously evaluate recovery capabilities, we introduce SPPE—the first benchmark dataset specifically designed for privacy recovery assessment—and develop a multimodal-guided generation framework that exploits complementary signals between protected proxy data and its edited variants to reconstruct privacy-sensitive content across diverse scenarios. Results: Experiments on SPPE and InstructPix2Pix demonstrate that our approach achieves strong privacy guarantees while significantly improving reconstruction fidelity and cross-task generalization. To our knowledge, this is the first method to jointly optimize privacy controllability and model utility in MLLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Privacy leakage in Multimodal Large Language Models (MLLMs) has long been an intractable problem. Existing studies, though effectively obscure private information in MLLMs, often overlook the evaluation of the authenticity and recovery quality of user privacy. To this end, this work uniquely focuses on the critical challenge of how to restore surrogate-driven protected data in diverse MLLM scenarios. We first bridge this research gap by contributing the SPPE (Surrogate Privacy Protected Editable) dataset, which includes a wide range of privacy categories and user instructions to simulate real MLLM applications. This dataset offers protected surrogates alongside their various MLLM-edited versions, thus enabling the direct assessment of privacy recovery quality. By formulating privacy recovery as a guided generation task conditioned on complementary multimodal signals, we further introduce a unified approach that reliably reconstructs private content while preserving the fidelity of MLLM-generated edits. The experiments on both SPPE and InstructPix2Pix further show that our approach generalizes well across diverse visual content and editing tasks, achieving a strong balance between privacy protection and MLLM usability.

Problem

• Evaluates authenticity and recovery quality of user privacy in MLLMs
• Focuses on restoring surrogate-driven protected data in MLLM scenarios
• Introduces a method to reconstruct private content while preserving edit fidelity

Innovation

• Surrogate Privacy Protected Editable dataset for evaluation
• Privacy recovery as guided generation with multimodal signals
• Unified approach balancing privacy protection and MLLM usability