AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Models

2025-11-14

AI Summary

Multimodal large language models (MLLMs) face significant challenges in selectively forgetting fine-grained visual concepts—e.g., removing specific sensitive categories (e.g., “person” or “license plate”)—while preserving model performance on semantically similar yet non-target concepts, all under regulatory constraints such as the “right to be forgotten.” Method: We propose AUVIC, the first framework systematically addressing fine-grained visual concept unlearning in MLLMs. It employs adversarial perturbation and gradient-based optimization to isolate and erase target concepts without full model retraining. Contribution/Results: We introduce VCUBench—the first benchmark for group-level visual concept unlearning—and demonstrate that AUVIC achieves state-of-the-art target-concept forgetting rates while inducing minimal degradation (<1.5% average accuracy drop) on non-target concepts. AUVIC significantly outperforms existing unlearning methods in both efficacy and generalization preservation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Multimodal Large Language Models (MLLMs) achieve impressive performance once optimized on massive datasets. Such datasets often contain sensitive or copyrighted content, raising significant data privacy concerns. Regulatory frameworks mandating the'right to be forgotten'drive the need for machine unlearning. This technique allows for the removal of target data without resource-consuming retraining. However, while well-studied for text, visual concept unlearning in MLLMs remains underexplored. A primary challenge is precisely removing a target visual concept without disrupting model performance on related entities. To address this, we introduce AUVIC, a novel visual concept unlearning framework for MLLMs. AUVIC applies adversarial perturbations to enable precise forgetting. This approach effectively isolates the target concept while avoiding unintended effects on similar entities. To evaluate our method, we construct VCUBench. It is the first benchmark designed to assess visual concept unlearning in group contexts. Experimental results demonstrate that AUVIC achieves state-of-the-art target forgetting rates while incurs minimal performance degradation on non-target concepts.

Problem

• Removing sensitive visual concepts from MLLMs without retraining
• Precisely forgetting target concepts while preserving related entities
• Addressing data privacy concerns through visual concept unlearning

Innovation

• Adversarial perturbations enable precise visual concept forgetting
• Isolates target concepts without affecting similar entities
• Benchmark VCUBench assesses unlearning in group contexts