Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation

2024-12-19

arXiv.org

AI Summary

To address the challenges of fine-grained control and poor adaptability across diverse body shapes in complex garment pattern generation, this paper proposes a multimodal controllable sewing pattern generation method. Methodologically, it introduces a novel two-stage diffusion training paradigm that jointly conditions generation on textual descriptions, 3D human shape parameters, and hand-drawn sketches. It further designs a sewing-pattern vector expansion mechanism and a compact latent space modeling strategy to enable efficient representation and synthesis of vector-based cutting patterns. Key technical contributions include: (1) multimodal conditional embedding alignment, (2) latent space compression optimization, and (3) sewing-structure-aware vector representation learning. Experiments demonstrate that our method significantly outperforms existing approaches in style complexity, anatomical fit, and editing controllability, enabling high-fidelity, editable, and cross-body-type cutting pattern generation.

Technology Category

• Natural Language Processing: Generation
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Generating sewing patterns in garment design is receiving increasing attention due to its CG-friendly and flexible-editing nature. Previous sewing pattern generation methods have been able to produce exquisite clothing, but struggle to design complex garments with detailed control. To address these issues, we propose SewingLDM, a multi-modal generative model that generates sewing patterns controlled by text prompts, body shapes, and garment sketches. Initially, we extend the original vector of sewing patterns into a more comprehensive representation to cover more intricate details and then compress them into a compact latent space. To learn the sewing pattern distribution in the latent space, we design a two-step training strategy to inject the multi-modal conditions, ie, body shapes, text prompts, and garment sketches, into a diffusion model, ensuring the generated garments are body-suited and detail-controlled. Comprehensive qualitative and quantitative experiments show the effectiveness of our proposed method, significantly surpassing previous approaches in terms of complex garment design and various body adaptability. Our project page: https://shengqiliu1.github.io/SewingLDM.

Problem

• Generating complex sewing patterns with detailed control
• Integrating multi-modal conditions for garment design
• Improving body adaptability in generated sewing patterns

Innovation

• Multimodal generative model for sewing patterns
• Two-step training strategy for condition injection
• Latent space compression for intricate details