Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation

2026-06-17

AI Summary

This work addresses the tendency of multimodal large language models to rely on textual shortcuts during online policy self-distillation, which undermines their grounding in visual content. To mitigate this issue, the authors propose ViGOS, a novel framework that explicitly decouples visual perception from logical reasoning in multimodal self-distillation for the first time. The student model operates in two stages: first generating visual descriptions and then performing answer reasoning, supervised respectively by a perception teacher (using image-only inputs) and a reasoning teacher (with full multimodal access), while a reference teacher provides holistic guidance. This three-teacher, multi-stage supervision mechanism effectively suppresses shortcut learning, significantly enhancing the model’s reliance on and robustness to visual input across multiple vision-language and reasoning benchmarks, all while preserving the performance benefits of self-distillation.

Abstract

On-policy self-distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense token-level targets conditioned on a reference target. This works well for LLM reasoning, but a direct extension to multimodal large language models (MLLMs) can create a shortcut: the privileged target may guide tokens mainly based on the text reference target rather than the image. We propose ViGOS, a visually grounded OPSD framework for MLLM post-training. The student first writes a visual description and then reasons toward the final answer. For valid rollouts, an image-only perception teacher supervises the description, while a privileged reasoning teacher supervises the reasoning and final answer on the same student prefix. A reference teacher is used only for invalid rollouts to recover the output format. Across general vision-language, expert reasoning, visual math, spatial grounding, and visual-language-prior benchmarks, ViGOS keeps the main benefits of OPSD and improves image-grounded behavior in shortcut-prone settings.

Problem

• multimodal large language models
• on-policy self-distillation
• shortcut learning
• visual grounding
• perception-reasoning decoupling

Innovation

• on-policy self-distillation
• multimodal large language models
• perception-reasoning decoupling
• visual grounding
• shortcut mitigation