DiT-Garment: Garment Dynamics with Diffusion Transformers

2026-09-16

AI Summary

本文提出DiT-Garment模型，利用2D扩散变压器架构在UV空间中学习3D变形，以解决动态3D服装建模问题，适用于任意设计和物理材质。

Abstract

We present DiT-Garment to model dynamic 3D clothing over human body models in arbitrary motion. Unlike existing methods, DiT-Garment can animate garments with unseen designs and physical materials, while allowing for direct inference of deformations for any target pose. To achieve this, we leverage a 2D diffusion transformer architecture to learn 3D deformations in a 2D UV-space. As the result is non-deterministic, our generative model learns the distribution of possible outcomes. The template garment is represented as a 3D triangle mesh spatially aligned with a 3D human body model in a standardized pose. To work with different garment designs without the need of a common template or complex graph convolution operations, the diffusion transformer is conditioned on a 3D position map of the template, represented in UV-space, which allows to implicitly learn a deformation of the 3D space around the body in standard pose. Further conditioning on body motion and physical parameters allows to physically ground the model. We quantitatively and qualitatively evaluate DiT-Garment on both synthetic and real data. While only trained on synthetic simulations of automatically generated cloth designs, our method generalizes to captured and artist-made garment designs. Code and data are available for research purposes at https://dumoulina.github.io/dit-garment/.

Problem

• dynamic 3D clothing
• arbitrary motion
• unseen designs
• physical materials
• deformations

Innovation

• Diffusion Transformers
• 3D Garment Animation
• UV-space Deformation Learning
• Physical Parameter Conditioning