The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning

2026-07-13

AI Summary

This work addresses the limited evidence-driven reasoning capability of current vision-language models (VLMs), which often stems from unstable visual evidence utilization during language generation. Through mechanistic analysis, the study uncovers a three-stage dynamic pattern in multimodal attention and, for the first time, identifies and formalizes a critical intermediate “Visual Relay Window” (VRW). Building on this insight, the authors propose TRACE, a framework that adaptively allocates and preserves visual information throughout the reasoning process. Integrating interpretable analysis, a lightweight trainable module, and a dynamic control strategy, TRACE is compatible with diverse open-source VLM architectures. Evaluated across four backbone models and seven benchmarks, it achieves an average improvement of 4.33 points (up to +6.6 points), significantly enhancing performance on complex and evidence-sensitive reasoning tasks.

Abstract

Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack, weakening evidence-grounded reasoning. To understand this fragility, we examine the internal dynamics of VLMs through a mechanistic lens and uncover a stable three-stage redistribution of multimodal attention focus across depth: an early question-conditioned organization, a critical middle visual-dominant relay, and a late return to answer formation. We operationalize the middle phase as the Visual Relay Window (VRW), and show that its geometry varies with task demand, is causally tied to grounded generation, and distinguishes unsupported answers from stronger reasoning trajectories. Guided by this internal rhythm, we propose TRACE, a task-adaptive inference-time control framework with lightweight trained modules. It reshapes relay allocation during prefill and preserves assembled visual support after handoff during decoding. Across four open-weight VLM backbones and seven benchmarks, TRACE delivers large gains on grounding-sensitive settings, improving them by 4.33 points on average and by up to 6.6 points, while also improving reasoning-heavy tasks. These results show that explicitly controlling multimodal focus across depth offers a unified and effective mechanism for strengthening evidence-grounded multimodal reasoning.

Problem

• vision-language models
• multimodal reasoning
• visual grounding
• attention dynamics
• evidence instability

Innovation

• Visual Relay Window
• multimodal attention dynamics
• evidence-grounded reasoning
• inference-time control
• TRACE