DeClotH: Decomposable 3D Cloth and Human Body Reconstruction from a Single Image

2025-03-25

AI Summary

To address the geometric and textural reconstruction challenges posed by severe clothing–body occlusion in single-image human digitization, this paper proposes the first decoupled 3D clothing–body reconstruction framework. Methodologically: (1) it establishes separate, differentiable 3D representations for clothing and body; (2) introduces a 3D-template-based geometric regularization to enforce structural plausibility; (3) designs a diffusion-guided appearance modeling module that integrates text-to-image priors with a clothing-specific diffusion model; and (4) implements a multi-stage joint optimization of geometry and appearance. Extensive evaluations on multiple benchmarks demonstrate significant improvements in reconstruction fidelity—both quantitatively and qualitatively surpassing state-of-the-art methods. To our knowledge, this is the first approach enabling high-fidelity, fully decoupled 3D dressed-human reconstruction from a single image.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Most existing methods of 3D clothed human reconstruction from a single image treat the clothed human as a single object without distinguishing between cloth and human body. In this regard, we present DeClotH, which separately reconstructs 3D cloth and human body from a single image. This task remains largely unexplored due to the extreme occlusion between cloth and the human body, making it challenging to infer accurate geometries and textures. Moreover, while recent 3D human reconstruction methods have achieved impressive results using text-to-image diffusion models, directly applying such an approach to this problem often leads to incorrect guidance, particularly in reconstructing 3D cloth. To address these challenges, we propose two core designs in our framework. First, to alleviate the occlusion issue, we leverage 3D template models of cloth and human body as regularizations, which provide strong geometric priors to prevent erroneous reconstruction by the occlusion. Second, we introduce a cloth diffusion model specifically designed to provide contextual information about cloth appearance, thereby enhancing the reconstruction of 3D cloth. Qualitative and quantitative experiments demonstrate that our proposed approach is highly effective in reconstructing both 3D cloth and the human body. More qualitative results are provided at https://hygenie1228.github.io/DeClotH/.

Problem

• Separately reconstructs 3D cloth and human body from single image
• Addresses occlusion challenges between cloth and human body
• Enhances cloth reconstruction with specialized diffusion model

Innovation

• Uses 3D template models for regularization
• Introduces cloth-specific diffusion model
• Separately reconstructs cloth and human body