Knowledge Vector Weakening: Efficient Training-free Unlearning for Large Vision-Language Models

2026-01-29

AI Summary

Large vision-language models face significant risks of privacy leakage and harmful content generation, necessitating efficient data unlearning mechanisms. This work proposes a training-free unlearning method that achieves precise data removal by directly intervening in the knowledge vectors within the model’s internal activations, thereby attenuating their contribution to the output. To the best of our knowledge, this is the first approach to enable full-model, intervention-based unlearning without gradient computation. By integrating knowledge vector identification with a training-free intervention strategy, the method substantially improves computational efficiency while preserving the performance on retained knowledge. Experimental results on the MLLMU and CLEAR benchmarks demonstrate that the proposed approach outperforms existing gradient-based and LoRA-based methods in both the unlearning–retention trade-off and computational efficiency.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large Vision-Language Models (LVLMs) are widely adopted for their strong multimodal capabilities, yet they raise serious concerns such as privacy leakage and harmful content generation. Machine unlearning has emerged as a promising solution for removing the influence of specific data from trained models. However, existing approaches largely rely on gradient-based optimization, incurring substantial computational costs for large-scale LVLMs. To address this limitation, we propose Knowledge Vector Weakening (KVW), a training-free unlearning method that directly intervenes in the full model without gradient computation. KVW identifies knowledge vectors that are activated during the model's output generation on the forget set and progressively weakens their contributions, thereby preventing the model from exploiting undesirable knowledge. Experiments on the MLLMU and CLEAR benchmarks demonstrate that KVW achieves a stable forget-retain trade-off while significantly improving computational efficiency over gradient-based and LoRA-based unlearning methods.

Problem

• machine unlearning
• large vision-language models
• privacy leakage
• harmful content generation
• computational efficiency

Innovation

• training-free unlearning
• knowledge vector weakening
• large vision-language models
• machine unlearning
• computational efficiency