GarmentSketch: Large-scale Sketch-to-Fashion Benchmark

2026-06-11

AI Summary

This work addresses the scarcity of large-scale, high-quality sketch-text paired data that has hindered research in sketch-based fashion image generation. We introduce a novel dataset comprising 26,249 fashion sketches spanning 21 garment categories, each accompanied by fine-grained and semantically accurate textual descriptions generated through a multi-stage human-in-the-loop pipeline integrating multimodal large language models and manual verification. This dataset constitutes the first large-scale, meticulously annotated sketch-text corpus tailored for sketch-guided text-to-image generation. Leveraging this resource, we conduct benchmark experiments that elucidate both the capabilities and limitations of current generative models on this task, thereby establishing a solid foundation for advancing sketch understanding and human-AI collaborative creative design.

Abstract

Fashion sketching is a cornerstone of design workflows, allowing rapid visualization of creative concepts prior to physical prototyping. Yet, progress in sketch-based fashion image synthesis has been hindered by the absence of large-scale, high-quality paired resources. To bridge this gap, we present GarmentSketch, a novel dataset comprising 26,249 fashion sketches across 21 garment categories, each paired with detailed textual descriptions. Captions were produced through a multi-stage pipeline that integrates multiple multimodal large language models (MLLMs) with human-in-the-loop refinement, ensuring both semantic accuracy and descriptive richness. We benchmark GarmentSketch on state-of-the-art generative models, providing baseline performance for sketch-guided text-to-image generation. Our experiments reveal both the promise and the current limitations of existing methods. By offering a comprehensive and richly annotated resource, GarmentSketch establishes a foundation for advancing sketch understanding, fine-grained fashion image generation, and creative human-AI collaboration in design. The dataset will be available at: https://khangbdd.github.io/garmentsketch.

Problem

• fashion sketch
• image synthesis
• large-scale dataset
• sketch-to-image
• generative models

Innovation

• sketch-to-image generation
• fashion dataset
• multimodal large language models
• human-in-the-loop annotation
• text-to-image synthesis