Compressing Image Style Training into a Single Model Forward

2026-06-11

AI Summary

Existing image stylization methods struggle to balance inference efficiency and style fidelity: adapter-based approaches often lose style specificity, while personalization techniques such as LoRA require per-style training. This work proposes i2L, a framework that, for the first time, fully feed-forward generates stylized LoRA weights—leveraging an image encoder, learnable LoRA queries, and a compact decoder head to predict text-to-image model LoRA parameters directly from one or multiple reference images, enabling optimization-free instant style instantiation. The method supports asymmetric classifier-free guidance, multi-style fusion, and seamless integration with controllable generation modules, effectively suppressing content copying while preserving prompt alignment. Experiments demonstrate that i2L significantly outperforms existing approaches on Z-Image, FLUX.2, and Hidream-O1, achieving consistent improvements in style fidelity, prompt adherence, and perceptual quality.

Abstract

Diffusion-based style transfer must balance inference efficiency with stylization fidelity. Adapter-based methods are efficient, but they inject style as an external condition and can either weaken reference-specific appearance or copy reference semantics into the generated image. Optimization-based personalization methods such as LoRA internalize style more effectively, but require a separate training process for every new style. We introduce i2L (image-to-LoRA), a framework that amortizes style LoRA training into a single forward pass. Given one or more reference images, i2L predicts LoRA weights for a text-to-image model, enabling immediate style instantiation without per-style optimization. The architecture combines an image encoder, learnable LoRA queries, and compressed decoding heads that generate adapted matrices. Training on semantically diverse style pairs encourages the predictor to preserve appearance cues while suppressing reference-content copying. Experiments on Z-Image, FLUX.2, and Hidream-O1 show that i2L improves style fidelity, prompt alignment, and perceptual quality over existing baselines. Because i2L produces explicit LoRA weights, it also supports asymmetric classifier-free guidance, multi-reference style fusion, and composition with controllable-generation modules.

Problem

• style transfer
• diffusion models
• LoRA
• personalization
• image generation

Innovation

• image-to-LoRA
• style transfer
• diffusion models
• parameter-efficient tuning
• single forward inference