CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models

2025-07-18

AI Summary

This work addresses the single-image content-style decomposition (CSD) problem, aiming for high-fidelity content extraction and controllable style transfer. We propose a scale-aware disentanglement framework grounded in visual autoregressive modeling: (i) a scale-aware alternating optimization strategy to strengthen content-style separation; (ii) an SVD-driven style rectification module to suppress content leakage into style representations; and (iii) an enhanced key-value memory mechanism to improve identity consistency across stylized outputs. To further boost generalization, we introduce scale-aligned training and targeted data augmentation. Additionally, we release CSD-100—the first dedicated benchmark for CSD evaluation. Extensive experiments demonstrate that our method achieves significant improvements over state-of-the-art approaches in both content fidelity and style consistency, enabling more flexible and precise visual synthesis with enhanced controllability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Personalized content ranking and presentation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Disentangling content and style from a single image, known as content-style decomposition (CSD), enables recontextualization of extracted content and stylization of extracted styles, offering greater creative flexibility in visual synthesis. While recent personalization methods have explored the decomposition of explicit content style, they remain tailored for diffusion models. Meanwhile, Visual Autoregressive Modeling (VAR) has emerged as a promising alternative with a next-scale prediction paradigm, achieving performance comparable to that of diffusion models. In this paper, we explore VAR as a generative framework for CSD, leveraging its scale-wise generation process for improved disentanglement. To this end, we propose CSD-VAR, a novel method that introduces three key innovations: (1) a scale-aware alternating optimization strategy that aligns content and style representation with their respective scales to enhance separation, (2) an SVD-based rectification method to mitigate content leakage into style representations, and (3) an Augmented Key-Value (K-V) memory enhancing content identity preservation. To benchmark this task, we introduce CSD-100, a dataset specifically designed for content-style decomposition, featuring diverse subjects rendered in various artistic styles. Experiments demonstrate that CSD-VAR outperforms prior approaches, achieving superior content preservation and stylization fidelity.

Problem

• Disentangling content and style in single images
• Enhancing creative flexibility in visual synthesis
• Improving content preservation and stylization fidelity

Innovation

• Scale-aware optimization for content-style separation
• SVD-based method to prevent content leakage
• Augmented K-V memory for identity preservation