Obliviate: Erasing Concepts from Autoregressive Image Generation Models

2026-06-26

AI Summary

This work addresses the lack of effective methods for erasing harmful content—such as explicit, violent, or branded imagery—from autoregressive image generation models. It introduces, for the first time, a guidance-based concept erasure framework that optimizes over the full autoregressive generation trajectory by supervising the visual token distribution via KL divergence. To ensure precise and stable erasure, the method incorporates an aligned visual prefix to construct a consistent target distribution. Extensive experiments on state-of-the-art models—including Liquid, Emu3-Gen, and Janus-Pro—demonstrate that the proposed approach reduces the generation rate of explicit content from 91.58% to 3.15% on the RAB benchmark, substantially outperforming existing techniques while preserving the models’ general-purpose generative capabilities.

Abstract

The widespread adoption of generative AI models has intensified concerns about misuse, including the creation of unsafe or disturbing imagery. To mitigate such issues, several concept erasure approaches have been proposed to remove harmful content from multimodal generative models. Yet concept erasure for autoregressive image generation remains largely unexplored, despite the growing relevance of these models in recent trends toward unified multimodal architectures. In this work, we fill this gap by introducing Obliviate, a guidance-based concept erasure method for autoregressive image generation. Our method builds on three key design choices: KL-based supervision over visual token distributions, trajectory-level updates over full autoregressive rollouts, and aligned visual prefixes for stable target construction. We evaluate Obliviate on three state-of-the-art autoregressive text-to-image models, Liquid, Emu3-Gen, and Janus-Pro, covering the erasure of explicit content, graphic violence, and branded imagery. Obliviate consistently outperforms current alternatives, reducing nudity on the defensive RAB benchmark from 91.58 to 3.15 while preserving overall model utility.

Problem

• concept erasure
• autoregressive image generation
• generative AI safety
• harmful content removal

Innovation

• concept erasure
• autoregressive image generation
• KL-based supervision
• trajectory-level update
• visual prefix alignment