Coarse-to-Fine Structure-Aware Artistic Style Transfer

2023-01-10

Applied Sciences

AI Summary

Existing artistic style transfer methods often neglect local structural alignment, leading to structural mismatches between content and style. To address this, we propose a two-stage structure-aware framework featuring a novel Structure-Selective Fusion (SSF) module. In the coarse-grain, low-resolution branch, the framework models global style distributions and structural priors; in the fine-grain, high-resolution branch, three cascaded SSF modules jointly transfer texture, color, and local geometric structure. Crucially, our method explicitly encodes and fuses local structural information from the style image, significantly improving structural consistency between the content and style domains. Quantitative and qualitative evaluations demonstrate that our approach outperforms multiple state-of-the-art methods, yielding high-resolution stylized images with superior visual quality and enhanced structural fidelity.

Technology Category

• Machine Learning: Structured Learning
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Web Mining and Content Analysis: Bridging structured and unstructured data
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Artistic style transfer aims to use a style image and a content image to synthesize a target image that retains the same artistic expression as the style image while preserving the basic content of the content image. Many recently proposed style transfer methods have a common problem; that is, they simply transfer the texture and color of the style image to the global structure of the content image. As a result, the content image has a local structure that is not similar to the local structure of the style image. In this paper, we present an effective method that can be used to transfer style patterns while fusing the local style structure to the local content structure. In our method, different levels of coarse stylized features are first reconstructed at low resolution using a coarse network, in which style color distribution is roughly transferred, and the content structure is combined with the style structure. Then, the reconstructed features and the content features are adopted to synthesize high-quality structure-aware stylized images with high resolution using a fine network with three structural selective fusion (SSF) modules. The effectiveness of our method is demonstrated through the generation of appealing high-quality stylization results and a comparison with some state-of-the-art style transfer methods.

Problem

• Enhancing local structure similarity
• Fusing style and content structures
• Generating high-resolution stylized images

Innovation

• Coarse-to-Fine network structure
• Structural Selective Fusion modules
• Local style and content integration