MLLM Machine Unlearning via Visual Knowledge Distillation

2025-12-12

AI Summary

This work addresses the challenge of targeted forgetting of sensitive visual knowledge in multimodal large language models (MLLMs), proposing a knowledge-decoupled forgetting method that selectively erases target visual representations without degrading textual capabilities. Methodologically, it introduces: (i) the first visual knowledge distillation (VKD) paradigm supervised by intermediate-layer visual representations; (ii) a modular fine-tuning strategy for visual modules to enable efficient, lightweight forgetting; and (iii) the first evaluation framework for relearning-attack robustness specifically designed for MLLMs. Extensive experiments across multiple state-of-the-art MLLMs demonstrate that the proposed method achieves a 23.6% higher forgetting success rate than prior art, accelerates inference by 3.2×, and exhibits显著 robustness against relearning attacks—thereby advancing both the efficacy and security of visual knowledge forgetting in MLLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recently, machine unlearning approaches have been proposed to remove sensitive information from well-trained large models. However, most existing methods are tailored for LLMs, while MLLM-oriented unlearning remains at its early stage. Inspired by recent studies exploring the internal mechanisms of MLLMs, we propose to disentangle the visual and textual knowledge embedded within MLLMs and introduce a dedicated approach to selectively erase target visual knowledge while preserving textual knowledge. Unlike previous unlearning methods that rely on output-level supervision, our approach introduces a Visual Knowledge Distillation (VKD) scheme, which leverages intermediate visual representations within the MLLM as supervision signals. This design substantially enhances both unlearning effectiveness and model utility. Moreover, since our method only fine-tunes the visual components of the MLLM, it offers significant efficiency advantages. Extensive experiments demonstrate that our approach outperforms state-of-the-art unlearning methods in terms of both effectiveness and efficiency. Moreover, we are the first to evaluate the robustness of MLLM unlearning against relearning attacks.

Problem

• Remove sensitive visual knowledge from MLLMs
• Preserve textual knowledge during unlearning
• Enhance unlearning effectiveness and model utility

Innovation

• Disentangle visual and textual knowledge for selective erasure
• Use intermediate visual representations as supervision signals
• Only fine-tune visual components for efficiency advantages