Image2Garment: Simulation-ready Garment Generation from a Single Image

2026-01-14

AI Summary

This work addresses the challenge of generating physically plausible, simulatable garments from a single image, which is hindered by data scarcity and the ill-posed nature of the problem. The authors propose a feedforward, end-to-end framework that fine-tunes a vision-language model to infer fabric composition and material attributes, then maps these predictions to physical simulation parameters via a lightweight predictor—eliminating the need for iterative optimization or multi-view inputs. To support this approach, they introduce FTAG and T2P, the first datasets aligning real-world garment images with corresponding physical parameters, thereby removing reliance on differentiable simulation pipelines. Experimental results demonstrate that the method outperforms existing image-to-garment generation techniques in fabric estimation, attribute prediction, and simulation fidelity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Estimating physically accurate, simulation-ready garments from a single image is challenging due to the absence of image-to-physics datasets and the ill-posed nature of this problem. Prior methods either require multi-view capture and expensive differentiable simulation or predict only garment geometry without the material properties required for realistic simulation. We propose a feed-forward framework that sidesteps these limitations by first fine-tuning a vision-language model to infer material composition and fabric attributes from real images, and then training a lightweight predictor that maps these attributes to the corresponding physical fabric parameters using a small dataset of material-physics measurements. Our approach introduces two new datasets (FTAG and T2P) and delivers simulation-ready garments from a single image without iterative optimization. Experiments show that our estimator achieves superior accuracy in material composition estimation and fabric attribute prediction, and by passing them through our physics parameter estimator, we further achieve higher-fidelity simulations compared to state-of-the-art image-to-garment methods.

Problem

• simulation-ready garment
• single image
• material properties
• physical accuracy
• garment generation

Innovation

• simulation-ready garment
• vision-language model
• fabric attribute estimation
• physics parameter prediction
• single-image garment reconstruction