IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation

2026-06-23

AI Summary

Existing multimodal large models struggle to precisely adhere to structural constraints—such as object count, spatial relationships, attribute binding, and coarse layout—in text-to-image generation. This work proposes an Implicit Visual Chain-of-Thought (IV-CoT) framework that decouples the generation process into two stages via a structure–semantics query separation architecture: first, a structure query implicitly derives a visual plan, and then a semantics query renders the appearance conditioned on this plan. The method innovatively introduces a sketch-based supervision mechanism during training, enabling structural–appearance disentanglement at inference time without requiring explicit sketches, and supports implicit visual reasoning within a single forward pass. Evaluated on GenEval and T2I-CompBench, the approach achieves state-of-the-art performance, with visualizations confirming the complementary roles of structure and semantics queries.

Abstract

Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream. To address this issue, we propose Implicit Visual Chain-of-Thought (IV-CoT), a latent visual reasoning framework for query-conditioned image generation. IV-CoT decomposes the visual conditioning queries into a structural-to-semantic cascade, where structural queries first form a latent visual plan and semantic queries then render appearance conditioned on this plan. To guide the structural queries, we introduce training-only sketch supervision, which encourages them to capture structure from sketches without requiring sketch extraction or intermediate decoding at inference time. IV-CoT performs implicit CoT reasoning in a single forward pass and achieves superior results on GenEval and T2I-CompBench. Visualizations and analyses demonstrate that the learned structural and semantic queries play complementary roles in structure-aware generation.

Problem

• structure-aware generation
• text-to-image generation
• spatial relations
• object counts
• attribute binding

Innovation

• Implicit Visual Chain-of-Thought
• structure-aware generation
• latent visual reasoning
• sketch supervision
• query decomposition