On Synthetic Texture Datasets: Challenges, Creation, and Curation

2024-09-16

arXiv.org

AI Summary

Texture research has long been hindered by the scarcity of high-quality, diverse benchmark data, impeding systematic evaluation of models on texture bias, interpretability, and robustness. To address this, we introduce the first large-scale text-to-texture synthesis framework and release the open-source PTD dataset—comprising 362,880 images across 56 texture categories. We systematically investigate the application of text-to-image diffusion models (e.g., Stable Diffusion) to texture generation, uncovering for the first time the unexpected sensitivity of NSFW filters to texture patterns. We further propose a scalable prompt-engineering strategy and a multi-stage automated filtering pipeline to mitigate such biases. Comprehensive evaluation—including quantitative metrics and human studies—confirms PTD’s high fidelity and strong diversity. The dataset is publicly available on Zenodo to foster community advancement in texture modeling and evaluation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The influence of textures on machine learning models has been an ongoing investigation, specifically in texture bias/learning, interpretability, and robustness. However, due to the lack of large and diverse texture data available, the findings in these works have been limited, as more comprehensive evaluations have not been feasible. Image generative models are able to provide data creation at scale, but utilizing these models for texture synthesis has been unexplored and poses additional challenges both in creating accurate texture images and validating those images. In this work, we introduce an extensible methodology and corresponding new dataset for generating high-quality, diverse texture images capable of supporting a broad set of texture-based tasks. Our pipeline consists of: (1) developing prompts from a range of descriptors to serve as input to text-to-image models, (2) adopting and adapting Stable Diffusion pipelines to generate and filter the corresponding images, and (3) further filtering down to the highest quality images. Through this, we create the Prompted Textures Dataset (PTD), a dataset of 362,880 texture images that span 56 textures. During the process of generating images, we find that NSFW safety filters in image generation pipelines are highly sensitive to texture (and flag up to 60% of our texture images), uncovering a potential bias in these models and presenting unique challenges when working with texture data. Through both standard metrics and a human evaluation, we find that our dataset is high quality and diverse. Our dataset is available for download at https://zenodo.org/records/15359142.

Problem

• Lack of large diverse texture datasets limits ML research
• Generating accurate synthetic texture images poses challenges
• NSFW filters show bias against texture image generation

Innovation

• Using text-to-image models for texture synthesis
• Developing prompts from diverse descriptors
• Filtering images for high-quality texture dataset