Does Multimodal Large Language Model Truly Unlearn? Stealthy MLLM Unlearning Attack

2025-06-09

arXiv.org

AI Summary

This work exposes a fundamental privacy vulnerability in multimodal large language model (MLLM) unlearning: “unlearned” sensitive information is not genuinely erased but remains systematically recoverable. To demonstrate this, we propose the Steganographic Unloading Attack (SUA) framework, which employs gradient-based optimization to learn a universal image perturbation—imperceptible at the pixel level—that adversarially reactivates internal MLLM representations and enforces cross-modal alignment between text and image embeddings. Our key innovation is an embedding alignment loss that jointly optimizes semantic robustness and pixel-level stealth. Experiments show that a single universal perturbation reliably restores unlearned content across diverse unlearned MLLMs and generalizes to unseen images. This is the first empirical evidence that MLLM unlearning achieves only superficial removal—not intrinsic forgetting—thereby undermining its claimed privacy guarantees.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multimodal Large Language Models (MLLMs) trained on massive data may memorize sensitive personal information and photos, posing serious privacy risks. To mitigate this, MLLM unlearning methods are proposed, which fine-tune MLLMs to reduce the ``forget''sensitive information. However, it remains unclear whether the knowledge has been truly forgotten or just hidden in the model. Therefore, we propose to study a novel problem of LLM unlearning attack, which aims to recover the unlearned knowledge of an unlearned LLM. To achieve the goal, we propose a novel framework Stealthy Unlearning Attack (SUA) framework that learns a universal noise pattern. When applied to input images, this noise can trigger the model to reveal unlearned content. While pixel-level perturbations may be visually subtle, they can be detected in the semantic embedding space, making such attacks vulnerable to potential defenses. To improve stealthiness, we introduce an embedding alignment loss that minimizes the difference between the perturbed and denoised image embeddings, ensuring the attack is semantically unnoticeable. Experimental results show that SUA can effectively recover unlearned information from MLLMs. Furthermore, the learned noise generalizes well: a single perturbation trained on a subset of samples can reveal forgotten content in unseen images. This indicates that knowledge reappearance is not an occasional failure, but a consistent behavior.

Problem

• Recovering unlearned sensitive information from multimodal LLMs after unlearning procedures
• Developing stealthy attack methods that bypass semantic detection mechanisms
• Investigating whether unlearning truly removes knowledge or just hides it

Innovation

• Learns universal noise pattern to trigger unlearned content
• Uses embedding alignment loss for semantic stealthiness
• Single perturbation generalizes to reveal forgotten knowledge