Color Me Correctly: Bridging Perceptual Color Spaces and Text Embeddings for Improved Diffusion Generation

2025-09-12

AI Summary

Current text-to-image (T2I) diffusion models struggle to accurately interpret fine-grained or compound color descriptions (e.g., “Tiffany blue”, “hot pink”), resulting in low color fidelity. To address this, we propose a lightweight, training-free, fine-tuning-free, and reference-free framework: first, leveraging a large language model (LLM) to disambiguate color terms and map them to the perceptually uniform CIELAB color space; second, embedding the parsed chromatic coordinates into the text encoding process to enable semantic-driven color mixing within the text embedding space. This is the first work to jointly model CIELAB space and optimize text embeddings for explicit color control in T2I generation. Experiments demonstrate significant improvements in color alignment accuracy across diverse complex color prompts, while preserving overall image quality—effectively bridging the semantic–visual color gap between textual descriptions and generated outputs.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Accurate color alignment in text-to-image (T2I) generation is critical for applications such as fashion, product visualization, and interior design, yet current diffusion models struggle with nuanced and compound color terms (e.g., Tiffany blue, lime green, hot pink), often producing images that are misaligned with human intent. Existing approaches rely on cross-attention manipulation, reference images, or fine-tuning but fail to systematically resolve ambiguous color descriptions. To precisely render colors under prompt ambiguity, we propose a training-free framework that enhances color fidelity by leveraging a large language model (LLM) to disambiguate color-related prompts and guiding color blending operations directly in the text embedding space. Our method first employs a large language model (LLM) to resolve ambiguous color terms in the text prompt, and then refines the text embeddings based on the spatial relationships of the resulting color terms in the CIELAB color space. Unlike prior methods, our approach improves color accuracy without requiring additional training or external reference images. Experimental results demonstrate that our framework improves color alignment without compromising image quality, bridging the gap between text semantics and visual generation.

Problem

• Resolving ambiguous color terms in text-to-image generation
• Improving color fidelity without training or reference images
• Bridging text semantics and visual color accuracy

Innovation

• LLM disambiguates color terms
• Refines embeddings via CIELAB space
• Training-free color blending framework