Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models

2026-09-08

AI Summary

研究通过引入生成性接地反馈机制，解决统一多模态模型中视觉理解和生成分离优化的问题，提高文本到图像的生成质量及视觉理解能力。

Abstract

Unified multimodal models integrate visual understanding and generation within a single network, yet the two capabilities are commonly optimized as separate tasks. We introduce Generative Grounding Feedback(GGF), a self-evolving post-training framework that uses only text prompts and the model's own visual experience. Given a prompt, the model first generates a visual ``dream.''Flow-level feedback compares text-, image-, and repair-conditioned predictions at the same noisy latent state, transferring image-grounded generation directions to the prompt condition. Dream replay grounding replays this dream through captioning and re-imagination, training claim-level evidence to remain consistent across the replay while separating unrelated visual experiences. Jointly optimized, these two directions let generation provide visual grounding for understanding and understanding refine subsequent generation without paired image--text supervision. Experiments across unified models with different understanding--generation integration designs show consistent improvements in text-to-image generation together with modest gains in visual understanding.

Problem

• Unified Multimodal Models
• Visual Understanding
• Generation
• Text-to-Image Generation

Innovation

• Generative Grounding Feedback
• self-evolving post-training framework
• dream replay grounding
• unified multimodal models