Dressing the Imagination: A Dataset for AI-Powered Translation of Text into Fashion Outfits and A Novel KAN Adapter for Enhanced Feature Adaptation

2024-11-21

arXiv.org

AI Summary

This work addresses two key challenges in text-to-fashion image generation: weak semantic alignment between textual descriptions and generated outfits, and low style fidelity. To this end, we introduce FLORA—the first professional-grade fashion text-outfit pairing dataset (4,330 samples), featuring designer-level fine-grained textual annotations. We further propose KAN Adapter, the first adapter architecture incorporating the Kolmogorov-Arnold Network (KAN) into diffusion model fine-tuning; it replaces LoRA by employing learnable spline-based activation functions to explicitly model complex, non-linear style-semantic relationships. Experiments on FLORA demonstrate that KAN Adapter significantly improves both style fidelity and text-outfit semantic alignment accuracy, while accelerating training convergence. All components—including the FLORA dataset, source code, and pretrained models—are publicly released.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Specialized datasets that capture the fashion industry's rich language and styling elements can boost progress in AI-driven fashion design. We present FLORA (Fashion Language Outfit Representation for Apparel Generation), the first comprehensive dataset containing 4,330 curated pairs of fashion outfits and corresponding textual descriptions. Each description utilizes industry-specific terminology and jargon commonly used by professional fashion designers, providing precise and detailed insights into the outfits. Hence, the dataset captures the delicate features and subtle stylistic elements necessary to create high-fidelity fashion designs. We demonstrate that fine-tuning generative models on the FLORA dataset significantly enhances their capability to generate accurate and stylistically rich images from textual descriptions of fashion sketches. FLORA will catalyze the creation of advanced AI models capable of comprehending and producing subtle, stylistically rich fashion designs. It will also help fashion designers and end-users to bring their ideas to life. As a second orthogonal contribution, we introduce KAN Adapters, which leverage Kolmogorov-Arnold Networks (KAN) as adaptive modules. They serve as replacements for traditional MLP-based LoRA adapters. With learnable spline-based activations, KAN Adapters excel in modeling complex, non-linear relationships, achieving superior fidelity, faster convergence and semantic alignment. Extensive experiments and ablation studies on our proposed FLORA dataset validate the superiority of KAN Adapters over LoRA adapters. To foster further research and collaboration, we will open-source both the FLORA and our implementation code.

Problem

• Creating AI models for translating text into fashion outfits.
• Developing a dataset with detailed fashion descriptions and outfits.
• Introducing KAN Adapters for better feature adaptation in AI models.

Innovation

• FLORA dataset for AI fashion design
• KAN Adapters for enhanced feature adaptation
• Open-source FLORA and implementation code