SPAST: Arbitrary style transfer with style priors via pre-trained large-scale model

2025-05-01

Neural Networks

AI Summary

Existing arbitrary style transfer methods face a fundamental trade-off: lightweight models yield low-fidelity outputs with prominent artifacts, whereas large models achieve higher visual quality but suffer from poor content-structure preservation and slow inference. This paper proposes a fine-tuning-free lightweight framework that, for the first time, embeds learnable explicit style priors into the frozen CLIP and diffusion Transformer (DiT) feature spaces, enabling effective content-style disentanglement. Our approach integrates CLIP-aligned guidance, DiT feature distillation, plug-and-play style adapters, and a contrastive style reconstruction loss—enabling zero-shot generalization to unseen styles and cross-domain reuse. On MSCOCO→WikiArt, our method reduces FID by 37%, improves style similarity by 2.1×, and achieves 48 fps inference speed at 512×512 resolution—significantly outperforming AdaIN, StyleCLIP, and LDM-Stylize.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Problem

• Improving quality of stylized images in style transfer
• Reducing inference time in large-scale model-based methods
• Preserving content structure while applying style features

Innovation

• Uses Local-global Window Size Stylization Module
• Incorporates style prior loss from large model
• Reduces inference time while preserving quality