POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking

2026-07-07

AI Summary

Current machine unlearning methods for multimodal large language models lack robustness against adversarial attacks and struggle to effectively prevent the recovery of privacy- or copyright-sensitive information. This work proposes POPS—a novel adversarial strategy that optimizes prompt suffixes to induce the model into generating plausible private samples, which are then used as synthetic data to fine-tune the model and reconstruct previously unlearned multimodal knowledge. POPS is the first to expose a fundamental vulnerability in existing unlearning mechanisms, achieving near-complete recovery of sensitive information across multiple benchmarks. These results starkly reveal the fragility of current unlearning algorithms and pose a serious challenge to the reliability of prevailing paradigms for multimodal privacy preservation.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on cross-modal tasks by jointly training on large-scale textual and visual data, where privacy-sensitive examples could be unintentionally encoded, raising concerns about privacy or copyright violation. To this end, Multi-modality Machine Unlearning (MMU) was proposed as a mitigation that can effectively force MLLMs to forget private information. However, the robustness of such unlearning methods is not fully exploited when the model is published and accessible to malicious users. In this paper, we propose a novel adversarial strategy, namely Prompt-Optimized Parameter Shaking (POPS), aiming to recover the supposedly unlearned multi-modality knowledge from the MLLMs. Our method elicits the victim MLLMs to generate potential private examples via prompt-suffix optimization, and then exploits these synthesized outputs to fine-tune the models so they disclose the true private information. The experiments on the different MMU benchmarks reveal substantial weaknesses in the existing MMU algorithms. Our POPS can even achieve a near-complete recovery of supposedly erased sensitive information on the unlearned MLLMs, exposing fundamental vulnerabilities that challenge the foundational robustness of representative MMU-based privacy protections.

Problem

• Multimodal Large Language Models
• Machine Unlearning
• Privacy
• Adversarial Attack
• Knowledge Recovery

Innovation

• Prompt-Optimized Parameter Shaking
• Multimodal Machine Unlearning
• Adversarial Recovery
• Privacy Vulnerability
• Multimodal Large Language Models