Rethinking Post-Unlearning Behavior of Large Vision-Language Models

2025-06-03

AI Summary

Large vision-language models (LVLMs) trained on web-sourced data pose significant privacy risks, yet existing machine unlearning methods often neglect post-unlearning response quality, leading to degradation, hallucination, or excessive refusal. Method: We propose PUBG—the first generative unlearning framework explicitly optimizing for “post-unlearning response quality.” PUBG jointly models vision-language constraints via contrastive learning and output distribution alignment, and introduces a controllable response guidance mechanism to enforce privacy safety, informativeness, and visual grounding in unlearned outputs. Contribution/Results: Experiments demonstrate that PUBG achieves zero privacy leakage while improving response accuracy on unlearned samples by 32% over baselines; it also attains state-of-the-art performance in visual relevance and information richness.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

Machine unlearning is used to mitigate the privacy risks of Large Vision-Language Models (LVLMs) arising from training on large-scale web data. However, existing unlearning methods often fail to carefully select substitute outputs for forget targets, resulting in Unlearning Aftermaths-undesirable behaviors such as degenerate, hallucinated, or excessively refused responses. We highlight that, especially for generative LVLMs, it is crucial to consider the quality and informativeness of post-unlearning responses rather than relying solely on naive suppression. To address this, we introduce a new unlearning task for LVLMs that requires models to provide privacy-preserving yet informative and visually grounded responses. We also propose PUBG, a novel unlearning method that explicitly guides post-unlearning behavior toward a desirable output distribution. Experiments show that, while existing methods suffer from Unlearning Aftermaths despite successfully preventing privacy violations, PUBG effectively mitigates these issues, generating visually grounded and informative responses without privacy leakage for forgotten targets.

Problem

• Mitigate privacy risks in Large Vision-Language Models post-unlearning
• Address undesirable behaviors like hallucinations in unlearning responses
• Ensure informative and visually grounded outputs after unlearning

Innovation

• Introduces privacy-preserving informative LVLM unlearning
• Proposes PUBG for desirable output distribution
• Ensures visually grounded responses without privacy leaks