Mixture of Style Experts for Diverse Image Stylization

2026-03-17

AI Summary

Existing image stylization methods predominantly rely on color transformations, often failing to simultaneously preserve semantic structure and material details. This work proposes StyleExpert, a novel framework that introduces the Mixture-of-Experts (MoE) mechanism into style transfer for the first time. By employing a unified style encoder to map diverse styles into a shared latent space and designing a similarity-aware gating network that dynamically routes inputs to specialized experts, StyleExpert enables multi-level style representation—from texture to deep semantics. Trained on large-scale content–style–output triplets using diffusion models, the method significantly outperforms existing approaches in preserving both semantic fidelity and fine-grained details, while demonstrating strong generalization to unseen styles.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Diffusion-based stylization has advanced significantly, yet existing methods are limited to color-driven transformations, neglecting complex semantics and material details.We introduce StyleExpert, a semantic-aware framework based on the Mixture of Experts (MoE). Our framework employs a unified style encoder, trained on our large-scale dataset of content-style-stylized triplets, to embed diverse styles into a consistent latent space. This embedding is then used to condition a similarity-aware gating mechanism, which dynamically routes styles to specialized experts within the MoE architecture. Leveraging this MoE architecture, our method adeptly handles diverse styles spanning multiple semantic levels, from shallow textures to deep semantics. Extensive experiments show that StyleExpert outperforms existing approaches in preserving semantics and material details, while generalizing to unseen styles. Our code and collected images are available at the project page: https://hh-lg.github.io/StyleExpert-Page/.

Problem

• image stylization
• semantic preservation
• material details
• diffusion models
• style diversity

Innovation

• Mixture of Experts
• semantic-aware stylization
• diffusion-based stylization
• style encoder
• diverse image stylization