Style-CCL: Content-Preserving Style Transfer via Curriculum Continual Learning

2026-06-04

AI Summary

Existing diffusion Transformers struggle to effectively disentangle content and style in style transfer, often being dominated by semantic-level style cues, which leads to insufficient texture learning and content distortion. To address this, this work proposes Style-CCL—the first multi-stage training framework that integrates curriculum learning with continual learning. It trains a dual-branch SC-DiT model following a structured progression from “semantic to texture” and “clean to synthetic” data, while incorporating random memory replay to mitigate catastrophic forgetting. The approach leverages independent RoPE embeddings, causal masking, and reverse triplets to construct a million-scale dataset. Evaluated on style similarity, content consistency, and aesthetic quality, Style-CCL achieves state-of-the-art performance, significantly enhancing both stylistic expressiveness and content fidelity.

Abstract

Content-Preserving Style transfer, given content and style references, remains challenging for Diffusion Transformers (DiTs) due to entangled content and style features. With a reverse triplet synthesis pipeline to build a million-scale training set and a dual-branch Style-Content DiT (SC-DiT) that decouples style and content via separate ROPE embeddings and causal masking, we observe that such a one-stage training paradigm on mixed style categories causes semantic styles to dominate, hindering texture style learning, and harming content preservation. To address these issues, we propose Style-CCL, a Multi-Stage Curriculum Continual Learning framework that trains SC-DiT from semantic (easy) to texture (hard) styles, and from clean to synthetic data, with Random Memory Rehearsal across stages to avoid catastrophic forgetting. Extensive experiments demonstrate that our Style-CCL achieves state-of-the-art performance in three core metrics: style similarity, content consistency, and aesthetic quality.

Problem

• Content-Preserving Style Transfer
• Diffusion Transformers
• Style-Content Entanglement
• Texture vs Semantic Styles
• Catastrophic Forgetting

Innovation

• Curriculum Continual Learning
• Diffusion Transformers
• Style-Content Disentanglement
• Memory Rehearsal
• Content-Preserving Style Transfer