SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Models

2026-06-14

AI Summary

Existing concept erasure methods, when directly applied to Vision Autoregressive (VAR) models, often induce semantic collapse and visual artifacts due to their neglect of the heterogeneous multi-scale generation mechanism inherent in VAR. This work reveals for the first time that target semantics are locked at Scale-0 in VAR, leading to the formulation of the "Semantic Singularity Axiom" and the development of Incremental Semantic Significance Analysis (ISSA). Building upon this insight, we propose SACE—the first scale-aware concept erasure framework—which confines intervention exclusively to the initial scale. By integrating an entropy-regularized erasure objective with a restorative preservation loss, SACE achieves precise, safe, and lightweight concept removal with negligible training overhead and minimal degradation in visual quality, thereby substantially mitigating safety risks in VAR models.

Abstract

The rapid progress of visual autoregressive (VAR) models has unlocked a transformative frontier for high-fidelity text-to-image synthesis, while heightening concerns over the safety alignment of generated content. Naive application of existing erasure techniques to VAR models causes catastrophic semantic collapse and visual artifacts, since they are predominantly designed for the homogeneous denoising steps of diffusion models. To address this foundational challenge, we first propose the Semantic Singularity Axiom, which posits that any target semantic concept embedded within a prompt is definitively locked at Scale-0. Then rigorously validate this axiom through our proposed Incremental Semantic Saliency Analysis (ISSA),which also enable the community to transparently inspect the coarse-to-fine semantic injection process. Guided by this insight, we introduce the first scale-aware concept erasure framework (SACE) for VAR models. By strictly confining interventions to the first scale, our approach couples an Entropy-Regularized Erasure Objective to prevent high-entropy sampling degeneration, alongside a restorative preservation loss to safely anchor the integrity of entangled benign priors. Extensive experiments demonstrate that our method achieves surgical concept erasure performance across various domains with minimal training overhead, timely and elegently resolute the critical safety vulnerabilities inherent in emerging VAR architectures. Code is available at: https://github.com/limerenceysy/SACE}{https://github.com/limerenceysy/SACE.

Problem

• concept erasure
• visual autoregressive models
• safety alignment
• semantic collapse
• text-to-image synthesis

Innovation

• Semantic Singularity
• Visual Autoregressive Models
• Concept Erasure
• Scale-aware Intervention
• Entropy-Regularized Erasure