Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning

2026-07-01

AI Summary

This work addresses a critical language-space bottleneck in multimodal large language models for visual reasoning: discrete tokenization discards fine-grained perceptual details, while continuous latent-space reasoning suffers from performance degradation due to distributional mismatch between training and inference caused by prior contamination from “answer leakage.” To resolve this, the paper introduces the Asymmetric Mutual Variational Learning (AMVL) framework, which formally characterizes this prior contamination issue for the first time. AMVL jointly optimizes forward and reverse KL divergences to prevent posterior collapse into inference-inaccessible regions, thereby enabling stable and perception-rich continuous latent reasoning. Evaluated on the BLINK benchmark, the method achieves an average improvement of 10.83 points, with gains up to 32.00 points on specific tasks, significantly outperforming existing discrete and continuous approaches.

Abstract

Multimodal Large Language Models (MLLMs) are often constrained by a language-space bottleneck, forcing complex visual reasoning into discrete tokens which can lose perceptual nuance. A promising alternative is continuous latent reasoning, where the goal is to discover implicit reasoning pathways that bridge the multimodal query and the final answer. However, this introduces a severe train-inference mismatch: a training-time posterior, conditioned on the ground-truth answer, can exploit answer-dependent shortcuts. Standard variational training then forces the inference-time prior to mimic a posterior that has access to information unavailable at test time, leading to poor performance. To address this, we propose Asymmetric Mutual Variational Learning (AMVL), a framework that resolves this mismatch via a bidirectional calibration objective. A forward KL divergence trains the target-agnostic prior to match the posterior, while a novel reverse KL divergence simultaneously regularizes the posterior, preventing it from collapsing into inference-incompatible regions and mitigating this ``answer leakage''. We provide theoretical analysis formalizing this leakage as prior contamination and prove that our dual-KL objective reduces it. We instantiate AMVL in a latent-integrated MLLM and show that it consistently outperforms strong discrete and latent-reasoning baselines, improving the average score on the complex BLINK benchmark by +10.83 and achieving gains of up to +32.00 on individual reasoning tasks, with analyses confirming improved latent-space stability.

Problem

• multimodal reasoning
• train-inference mismatch
• answer leakage
• continuous latent reasoning
• variational learning

Innovation

• Asymmetric Mutual Variational Learning
• continuous latent reasoning
• train-inference mismatch
• answer leakage
• multimodal large language models