Attending to Multimodal Generation One Token at a Time

2026-07-04

AI Summary

This work addresses the limited understanding of the dynamic mechanisms governing when multimodal large language models (MLLMs) invoke visual versus textual information during generation. It presents the first systematic analysis of token-by-token attention dynamics, employing attention trajectory tracing, causal masking interventions, and test-time modulation across multiple open-source models. The study reveals consistent patterns: visual attention peaks precisely at tokens semantically related to the image, while instruction tokens are revisited during task transitions. Building on these insights, the authors propose an attention-guided intervention strategy that significantly enhances performance on multimodal tasks. Furthermore, controlled perturbations expose a critical vulnerability—models readily degrade to relying solely on linguistic priors or generate cross-modal inconsistencies—thereby underscoring the pivotal role of attention mechanisms in coherent multimodal generation.

Abstract

Multimodal large language models (MLLMs) generate responses autoregressively, integrating visual and linguistic information in an evolving context. Prior work on interpretability has focused on individual layers and circuits (where), leaving the token-level dynamics of multimodal computation during generation (when) underexplored. We address this gap and study attention shifts as per semantic role; tracking model attention to image, text, instruction, and previously generated tokens, One Token at a Time (OTaT). We introduce multimodal tasks that require explicit switching between visual and textual context within a single response. Across two mainstream model families and four open-weight MLLMs of varying sizes, we establish consistent patterns: attention to image peaks at tokens requiring image-derived information, instruction tokens are revisited during task transitions, and attention to previously generated tokens increases as the generation progresses. Causal attention blocking interventions validate the functional role of these trends. We profile model behavior under disrupted attention and observe responses falling back to language priors, or exhibiting cross-modal leakage, denial, or recovery. Finally, informed of the attention dynamics through our novel analysis, we propose a simple test-time intervention to boost attention to the relevant modality at the right time, significantly improving multimodal task performance.

Problem

• multimodal generation
• attention dynamics
• token-level analysis
• multimodal large language models
• autoregressive generation

Innovation

• multimodal attention dynamics
• token-level analysis
• attention intervention
• multimodal large language models
• cross-modal generation