Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

2026-05-12

AI Summary

Existing visual implicit reasoning methods suffer from unstable intermediate visual evidence due to feature space mismatches. This work proposes the GAP paradigm, which systematically identifies and addresses the norm mismatch between visual latent variables and input embeddings for the first time. Through alignment mechanisms operating at three granularities—feature, context, and capability—the approach enhances the stability and effectiveness of multimodal large language model reasoning. The method incorporates a lightweight PCA-based alignment head for latent variables, verifiable auxiliary visual supervision, and a model-capability-aware selective supervision strategy. Evaluated on Qwen2.5-VL 7B, it achieves state-of-the-art performance in perception-reasoning integration, with intervention experiments confirming that the generated latent variables provide task-relevant visual signals significantly superior to placeholder baselines.

Abstract

Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume~\citep{xie2025mhc,li2026siamesenorm,team2026attention}. This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose \textbf{GAP}, a \textbf{G}ranular \textbf{A}lignment \textbf{P}aradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacity-guided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.

Problem

• visual latent reasoning
• feature-space mismatch
• multimodal large language models
• latent feedback instability

Innovation

• visual latent reasoning
• feature-space alignment
• multimodal large language models
• granular alignment paradigm
• PCA-aligned latent head