Steering Visual Generation in Unified Multimodal Models with Understanding Supervision

2026-05-07

AI Summary

This work addresses the limited synergy between understanding and generation modules in existing unified multimodal models, where their high degree of decoupling hinders mutual enhancement. To bridge this gap, the authors propose UNO, a comprehension-guided post-training framework that, for the first time, explicitly leverages understanding tasks—such as image captioning and visual regression—as supervisory signals during the generative process. By incorporating both semantic abstraction and structural detail objectives, UNO enables effective gradient flow from understanding to generation. Remarkably, with only lightweight post-training, the method achieves substantial performance gains in image generation and editing tasks, demonstrating that enhanced comprehension capabilities can significantly improve generative quality.

Abstract

Unified multimodal models are envisioned to bridge the gap between understanding and generation. Yet, to achieve competitive performance, state-of-the-art models adopt largely decoupled understanding and generation components. This design, while effective for individual tasks, weakens the connection required for mutual enhancement, leaving the potential synergy empirically uncertain. We propose to explicitly restore this synergy by introducing Understanding-Oriented Post-Training (UNO), a lightweight framework that treats understanding not only as a distinct task, but also a direct supervisory signal to steer generative representations. By incorporating objectives that encode semantic abstraction (captioning) and structural details (visual regression), we enable effective gradient flow from understanding to generation. Extensive experiments on image generation and editing demonstrate that understanding can serve as an effective catalyst for generation.

Problem

• multimodal models
• understanding-generation gap
• synergy
• visual generation
• unified modeling

Innovation

• Understanding-Oriented Post-Training
• multimodal generation
• understanding supervision
• unified multimodal models
• gradient steering