CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

2026-03-09

AI Summary

Existing chain-of-thought (CoT)-based text-to-image generation methods rely on natural language planning, which struggles to precisely control complex spatial layouts, structured visual elements, and dense textual content. To address this limitation, this work proposes CoCo, a novel framework that, for the first time, employs executable code as the CoT medium. CoCo generates structured sketches via code, renders them in a sandboxed environment, and subsequently refines the output through high-fidelity image editing. To support this approach, we introduce CoCo-10K, the first paired dataset of structured sketches and refined images. Extensive experiments demonstrate that CoCo achieves significant performance gains—improving by 68.83%, 54.8%, and 41.23% respectively—on three benchmarks including StructT2IBench, substantially outperforming both direct generation baselines and other CoT-enhanced methods.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integration of Chain-of-Thought (CoT) reasoning. However, existing CoT-based T2I methods largely rely on abstract natural-language planning, which lacks the precision required for complex spatial layouts, structured visual elements, and dense textual content. In this work, we propose CoCo (Code-as-CoT), a code-driven reasoning framework that represents the reasoning process as executable code, enabling explicit and verifiable intermediate planning for image generation. Given a text prompt, CoCo first generates executable code that specifies the structural layout of the scene, which is then executed in a sandboxed environment to render a deterministic draft image. The model subsequently refines this draft through fine-grained image editing to produce the final high-fidelity result. To support this training paradigm, we construct CoCo-10K, a curated dataset containing structured draft-final image pairs designed to teach both structured draft construction and corrective visual refinement. Empirical evaluations on StructT2IBench, OneIG-Bench, and LongText-Bench show that CoCo achieves improvements of +68.83%, +54.8%, and +41.23% over direct generation, while also outperforming other generation methods empowered by CoT. These results demonstrate that executable code is an effective and reliable reasoning paradigm for precise, controllable, and structured text-to-image generation. The code is available at: https://github.com/micky-li-hd/CoCo

Problem

• text-to-image generation
• Chain-of-Thought
• structured visual elements
• spatial layout
• executable code

Innovation

• Code-as-CoT
• executable code reasoning
• structured text-to-image generation
• deterministic draft rendering
• multimodal chain-of-thought