CLOTH-HUGS: Cloth Aware Human Gaussian Splatting

2026-04-17

AI Summary

Existing methods struggle to faithfully reconstruct dynamically dressed humans under complex deformations due to their coupled representation of the body and loose clothing. This work proposes the first decoupled neural rendering framework based on Gaussian splatting, explicitly modeling the body and garments as separate Gaussian layers within a shared canonical space. Realism is enhanced through SMPL skeleton-driven animation, linear blend skinning, as-rigid-as-possible (ARAP) regularization, and physics-based consistency constraints. By incorporating mesh-topology-aware initialization, mask supervision, and depth-aware multi-channel rendering, our approach significantly outperforms state-of-the-art methods—reducing LPIPS error by up to 28% across multiple benchmarks—while enabling high-quality, temporally coherent dynamic garment reconstruction at over 60 frames per second.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We present Cloth-HUGS, a Gaussian Splatting based neural rendering framework for photorealistic clothed human reconstruction that explicitly disentangles body and clothing. Unlike prior methods that absorb clothing into a single body representation and struggle with loose garments and complex deformations, Cloth-HUGS represents the performer using separate Gaussian layers for body and cloth within a shared canonical space. The canonical volume jointly encodes body, cloth, and scene primitives and is deformed through SMPL-driven articulation with learned linear blend skinning weights. To improve cloth realism, we initialize cloth Gaussians from mesh topology and apply physics-inspired constraints, including simulation-consistency, ARAP regularization, and mask supervision. We further introduce a depth-aware multi-pass rendering strategy for robust body-cloth-scene compositing, enabling real-time rendering at over 60 FPS. Experiments on multiple benchmarks show that Cloth-HUGS improves perceptual quality and geometric fidelity over state-of-the-art baselines, reducing LPIPS by up to 28% while producing temporally coherent cloth dynamics.

Problem

• clothed human reconstruction
• loose garments
• complex deformations
• cloth realism
• body-cloth disentanglement

Innovation

• Gaussian Splatting
• Cloth-Body Disentanglement
• Neural Rendering
• Physics-Inspired Constraints
• Real-Time Rendering