MAST: Mask-Guided Attention Mass Allocation for Training-Free Multi-Style Transfer

2026-04-14

AI Summary

This work addresses boundary artifacts, style instability, and structural inconsistency in multi-style transfer—issues arising from interference among style representations—by proposing a training-free framework. The method explicitly modulates content–style interactions within the attention mechanism of diffusion models through four novel components: layout-preserving query anchoring, logarithmic attention weight allocation, sharpness-aware temperature scaling, and discrepancy-aware detail injection. These modules jointly optimize the multi-style fusion process to enhance coherence and fidelity. Experimental results demonstrate that the proposed approach effectively eliminates boundary artifacts, preserves structural consistency, and maintains high textural fidelity and spatial coherence even as the number of styles increases.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Style transfer aims to render a content image with the visual characteristics of a reference style while preserving its underlying semantic layout and structural geometry. While recent diffusion-based models demonstrate strong stylization capabilities by leveraging powerful generative priors and controllable internal representations, they typically assume a single global style. Extending them to multi-style scenarios often leads to boundary artifacts, unstable stylization, and structural inconsistency due to interference between multiple style representations. To overcome these limitations, we propose MAST (Mask-Guided Attention Mass Allocation for Training-Free Multi-Style Transfer), a novel training-free framework that explicitly controls content-style interactions within the diffusion attention mechanism. To achieve artifact-free and structure-preserving stylization, MAST integrates four connected modules. First, Layout-preserving Query Anchoring prevents global layout collapse by firmly anchoring the semantic structure using content queries. Second, Logit-level Attention Mass Allocation deterministically distributes attention probability mass across spatial regions, seamlessly fusing multiple styles without boundary artifacts. Third, Sharpness-aware Temperature Scaling restores the attention sharpness degraded by multi-style expansion. Finally, Discrepancy-aware Detail Injection adaptively compensates for localized high-frequency detail losses by measuring structural discrepancies. Extensive experiments demonstrate that MAST effectively mitigates boundary artifacts and maintains structural consistency, preserving texture fidelity and spatial coherence even as the number of applied styles increases.

Problem

• multi-style transfer
• boundary artifacts
• structural inconsistency
• diffusion models
• style interference

Innovation

• mask-guided attention
• training-free
• multi-style transfer
• diffusion models
• attention mass allocation