TILDE: TILt-based Distributional Erasure for Concept Unlearning

2026-07-07

AI Summary

This work addresses the challenge of concept erasure in text-to-image diffusion models, where specific concepts—such as those involving privacy or copyright—must be effectively forgotten post-deployment without degrading generation quality or semantic coverage for benign prompts. The authors formulate concept forgetting as a distribution alignment problem and introduce an anchor-free, energy-tilted target distribution that explicitly characterizes the ideal post-forgetting generative behavior, suppressing target concepts while preserving the relative probability mass of benign samples. A residual ∇-GFlowNet is employed to learn the corresponding score correction, enabling distribution-level concept removal. Experiments demonstrate that the proposed method significantly outperforms existing approaches across object, artistic style, and character forgetting tasks, achieving superior forgetting efficacy while better maintaining generation diversity and distributional fidelity.

Abstract

Concept unlearning in text-to-image diffusion models is critical for safe and practical deployment: with rising privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must be able to suppress unwanted concepts after training. Existing methods often remove the target concept effectively, but practical unlearning also requires an equally fundamental property: the unlearned model should retain quality, diversity, and semantic coverage on benign generation. The gold standard is a retain-only model trained from scratch without the unwanted data. However, common erasure objectives do not specify which post-unlearning distribution should approximate this reference, leaving retention as an implicit consequence of the update rule. We propose TILDE, TILt-based Distributional Erasure, which formulates concept unlearning as a distributional alignment problem: the desired target is the minimum-deviation conditional distribution from the pretrained model under a forgetting constraint. This energy-tilted, anchor-free target suppresses concept-expressing images while preserving benign relative mass for each prompt. We instantiate this principle with residual $\nabla$-GFlowNet training, which learns the score correction induced by the forget energy relative to the pretrained diffusion model. Across objects, artistic styles, and characters, TILDE achieves strong forgetting while improving retention and distributional fidelity over prior baselines.

Problem

• concept unlearning
• text-to-image diffusion models
• distributional erasure
• privacy
• copyright

Innovation

• concept unlearning
• distributional alignment
• energy-based modeling
• diffusion models
• GFlowNet