FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling

2024-11-29

AI Summary

Existing learning-based methods suffer from inaccurate normalization when modeling loose garments (e.g., long skirts, oversized sleeves), as large body–garment distances induce geometric discontinuities and fragmentation, hindering faithful modeling of pose-dependent deformations. To address this, we propose a region-adaptive hybrid modeling framework: exposed regions are directly copied; tight-fitting regions are deformed via Linear Blend Skinning (LBS); and loose regions are modeled by a novel free-form, part-aware generator. This paradigm enables, for the first time, decoupled, part-aware synthesis of distant garments—overcoming LBS’s fundamental limitations in loose regions. Evaluated on a dedicated loose-garment benchmark, our method achieves state-of-the-art performance, significantly improving visual fidelity and geometric detail reconstruction, especially in challenging dynamic scenarios such as long-skirt motion.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Learning with Manifolds
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Achieving realistic animated human avatars requires accurate modeling of pose-dependent clothing deformations. Existing learning-based methods heavily rely on the Linear Blend Skinning (LBS) of minimally-clothed human models like SMPL to model deformation. However, they struggle to handle loose clothing, such as long dresses, where the canonicalization process becomes ill-defined when the clothing is far from the body, leading to disjointed and fragmented results. To overcome this limitation, we propose FreeCloth, a novel hybrid framework to model challenging clothed humans. Our core idea is to use dedicated strategies to model different regions, depending on whether they are close to or distant from the body. Specifically, we segment the human body into three categories: unclothed, deformed, and generated. We simply replicate unclothed regions that require no deformation. For deformed regions close to the body, we leverage LBS to handle the deformation. As for the generated regions, which correspond to loose clothing areas, we introduce a novel free-form, part-aware generator to model them, as they are less affected by movements. This free-form generation paradigm brings enhanced flexibility and expressiveness to our hybrid framework, enabling it to capture the intricate geometric details of challenging loose clothing, such as skirts and dresses. Experimental results on the benchmark dataset featuring loose clothing demonstrate that FreeCloth achieves state-of-the-art performance with superior visual fidelity and realism, particularly in the most challenging cases.

Problem

• Modeling pose-dependent clothing deformations for realistic human avatars.
• Handling loose clothing like long dresses in human modeling.
• Improving visual fidelity and realism in challenging clothed human models.

Innovation

• Hybrid framework for clothed human modeling
• Free-form generator for loose clothing
• Segmented body regions for targeted deformation