EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

2026-03-18

AI Summary

This work addresses the limitations of layout-to-image generation and image grounding when treated in isolation, as well as the optimization challenges arising from their joint training. To overcome these issues, the authors propose a unified framework that enables synergistic optimization of both tasks through shared token representations, explicit modeling of task duality, and cycle-consistency constraints. A novel progressive three-stage training strategy is introduced: parallel multi-task pretraining, joint fine-tuning of both tasks, and unsupervised consistency enhancement via GRPO-based reinforcement learning. The proposed method achieves state-of-the-art performance on both layout generation and image grounding benchmarks, substantially validating the efficacy and mutual benefits of cross-task complementary modeling.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this work, we present EchoGen, a unified framework for layout-to-image generation and image grounding, capable of generating images with accurate layouts and high fidelity to text descriptions (e.g., spatial relationships), while grounding the image robustly at the same time. We believe that image grounding possesses strong text and layout understanding abilities, which can compensate for the corresponding limitations in layout-to-image generation. At the same time, images generated from layouts exhibit high diversity in content, thereby enhancing the robustness of image grounding. Jointly training both tasks within a unified model can promote performance improvements for each. However, we identify that this joint training paradigm encounters several optimization challenges and results in restricted performance. To address these issues, we propose progressive training strategies. First, the Parallel Multi-Task Pre-training (PMTP) stage equips the model with basic abilities for both tasks, leveraging shared tokens to accelerate training. Next, the Dual Joint Optimization (DJO) stage exploits task duality to sequentially integrate the two tasks, enabling unified optimization. Finally, the Cycle RL stage eliminates reliance on visual supervision by using consistency constraints as rewards, significantly enhancing the model's unified capabilities via the GRPO strategy. Extensive experiments demonstrate state-of-the-art results on both layout-to-image generation and image grounding benchmarks, and reveal clear synergistic gains from optimizing the two tasks together.

Problem

• layout-to-image generation
• image grounding
• joint training
• optimization challenges
• unified model

Innovation

• cycle-consistent learning
• unified layout-image generation
• image grounding
• progressive training
• GRPO