Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning

2026-07-10

AI Summary

This work addresses the challenges in unified multimodal reasoning, where automatically generating intermediate visual steps often introduces erroneous evidence and incurs excessive computational overhead. To overcome this, the authors propose AdaViG, a novel method that leverages internal signals of generative intent and visual fidelity from a unified multimodal model to dynamically decide—early in the visual generation process—whether to terminate redundant or harmful steps. AdaViG enables adaptive visual gating without requiring additional training, by analyzing internal model activations and integrating early evaluation with dynamic halting strategies. This approach significantly enhances inference efficiency while preserving or even improving output quality: it achieves up to a 5.7% accuracy gain, reduces visual generation FLOPs by 25.0%–91.0%, and cuts inference latency by 15.4%–45.6%.

Abstract

Unified multimodal models (UMMs) with interleaved reasoning, which generate both textual and visual steps as part of intermediate reasoning traces, have demonstrated great potential for visual mathematical reasoning tasks. However, we identify a key insight in this paradigm: generating intermediate visual reasoning steps is not always beneficial and can even be harmful, as self-generated visual steps may introduce erroneous visual evidence that misleads subsequent reasoning. Moreover, frequently triggering visual steps during reasoning incurs substantial computational and memory overhead, degrading inference efficiency. To address these accuracy and efficiency challenges, we observe that the model's internal signals can indicate whether a visual step will benefit reasoning before the entire visual generation is completed. Specifically, this work identifies two internal signals: 1) Generation Intent, which reflects whether the model has a concrete textual plan for what to draw, and 2) Visual Fidelity, which measures whether the visual generation remains grounded in the original input image. Leveraging these internal signals, we propose AdaViG, a training-free adaptive visual gating method for unified multimodal reasoning. AdaViG dynamically evaluates each triggered visual step at an early visual generation stage and aborts it when both signals are weak, thereby preventing misleading visual evidence from entering the reasoning trace while avoiding unnecessary computation. Comprehensive experiments demonstrate that AdaViG improves accuracy by up to 5.7% while reducing visual generation FLOPs by 25.0%-91.0% and wall-clock latency by 15.4%-45.6%.

Problem

• multimodal reasoning
• visual generation
• reasoning efficiency
• visual evidence
• intermediate reasoning

Innovation

• adaptive visual gating
• unified multimodal reasoning
• generation intent
• visual fidelity
• training-free