Low-Barrier Dataset Collection with Real Human Body for Interactive Per-Garment Virtual Try-On

2025-06-12

AI Summary

Existing image-based virtual try-on methods suffer from single-view constraints, low real-time performance, and reliance on expensive robotic mannequins for garment-wise modeling—limiting natural human deformation capture and yielding inaccurate cloth-human alignment. Method: We propose a real-human-driven, low-barrier data collection paradigm for interactive, garment-wise try-on—eliminating specialized hardware while supporting natural poses and deformations. We design a lightweight hybrid human representation integrating simplified DensePose with multi-view pose guidance, enabling real-time, device-free human-cloth interaction and precise alignment. Further, we combine per-garment neural rendering with real-time image alignment optimization. Results: Experiments demonstrate significant improvements over state-of-the-art methods in both visual quality and temporal consistency. A user study confirms that our approach effectively enhances confidence in apparel purchasing decisions.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Existing image-based virtual try-on methods are often limited to the front view and lack real-time performance. While per-garment virtual try-on methods have tackled these issues by capturing per-garment datasets and training per-garment neural networks, they still encounter practical limitations: (1) the robotic mannequin used to capture per-garment datasets is prohibitively expensive for widespread adoption and fails to accurately replicate natural human body deformation; (2) the synthesized garments often misalign with the human body. To address these challenges, we propose a low-barrier approach for collecting per-garment datasets using real human bodies, eliminating the necessity for a customized robotic mannequin. We also introduce a hybrid person representation that enhances the existing intermediate representation with a simplified DensePose map. This ensures accurate alignment of synthesized garment images with the human body and enables human-garment interaction without the need for customized wearable devices. We performed qualitative and quantitative evaluations against other state-of-the-art image-based virtual try-on methods and conducted ablation studies to demonstrate the superiority of our method regarding image quality and temporal consistency. Finally, our user study results indicated that most participants found our virtual try-on system helpful for making garment purchasing decisions.

Problem

• High cost and inaccuracy of robotic mannequins for garment dataset collection
• Misalignment of synthesized garments with human body in virtual try-on
• Lack of real-time performance and limited views in existing methods

Innovation

• Real human bodies for dataset collection
• Hybrid person representation with DensePose
• Accurate garment-body alignment without devices