DealMaTe: Multi-Dimensional Material Transfer via Diffusion Transformer

2026-05-15

AI Summary

This work addresses the limitations of existing diffusion-based material transfer methods, which often rely on textual guidance or complex auxiliary networks, leading to high computational overhead and feature misalignment. The authors propose DealMaTe, a streamlined diffusion framework that eliminates the need for text prompts or reference images by leveraging depth, normal, and illumination cues for multidimensional material transfer. Central to this approach is a lightweight Multi-Dim 3D Shader LoRA module that enables compatible control without altering the base model weights. Additionally, the integration of shader-aware causal cross-attention with KV caching enhances both inference efficiency and output stability. Built upon a diffusion Transformer architecture, DealMaTe achieves high-fidelity, harmonious material transfer across diverse objects and lighting conditions while significantly reducing model complexity and computational cost.

Abstract

Recently, diffusion-based material transfer methods rely on image fine-tuning or complex architectures with auxiliary networks but face challenges such as text dependency, additional computational costs, and feature misalignment. To address these limitations, we propose \textbf{DealMaTe}, using \underline{\textbf{de}}pth, norm\underline{\textbf{a}}l, and \underline{\textbf{l}}ighting images for \underline{\textbf{ma}}terial \underline{\textbf{t}}ransf\underline{\textbf{e}}r. DealMaTe is a simplified diffusion framework that eliminates text guidance and reference networks. We design a lightweight 3D information injection method, Multi-Dim 3D Shader LoRA, which, without modifying the base model weights, enables compatible control conditions and achieves harmonious and stable results. Additionally, we optimize the attention mechanism with Shader Causal Mutual Attention and key-value (KV) caching to reduce inference latency caused by multiple conditions, improve computational efficiency, and achieve high-quality material transfer results with low architectural complexity. Extensive experiments covering a wide variety of objects and lighting conditions consistently demonstrate that DealMaTe achieves remarkable high-fidelity material transfer under arbitrary input materials. The code is available at https://github.com/haha-lisa/DealMaTe.

Problem

• material transfer
• diffusion models
• text dependency
• computational cost
• feature misalignment

Innovation

• Diffusion Transformer
• Material Transfer
• LoRA
• 3D Shader
• Efficient Attention