Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

2026-06-17

AI Summary

Existing unified multimodal models rely on generating “Visual Thoughts” (VTs) for spatial reasoning tasks, incurring substantial computational overhead with limited performance gains. This work proposes Visual-OPSD, a novel approach that leverages semantic information from VT generation trajectories—rather than rendered images—for cross-modal knowledge transfer. By employing a weight-sharing teacher–student architecture and trajectory-based, token-wise Jensen–Shannon divergence distillation, the method efficiently transfers multimodal reasoning capabilities from a teacher model to a text-only student model. Evaluated across nine benchmarks, Visual-OPSD achieves an average improvement of 3.40 percentage points, accelerates inference by 14.3×, and outperforms same-scale vision-language models by 63.83 percentage points on Visual Spatial Reasoning (VSP) tasks.

Abstract

Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly an order-of-magnitude inference cost from multi-step diffusion. We find this cost yields limited direct benefit. On ThinkMorph, removing or noising VTs barely changes accuracy across nine benchmarks. Once rendered, attention concentrates on the VT regardless of content. Yet a KL diagnostic shows that conditioning on a privileged VT trace shifts the model's completion distribution. This suggests the generation pathway encodes useful reasoning beyond the rendered pixels. Motivated by this gap, we propose Visual On-Policy Self-Distillation(Visual-OPSD). Teacher and student share identical weights but differ in context: the teacher sees privileged VTs while the student sees only the question. Token-level JSD distillation on on-policy student trajectories transfers the teacher's reasoning to a text-only student. Across nine benchmarks, Visual-OPSD improves over its generative teacher by $+3.40$pp with $14.3\times$ speedup (10.0s vs. 142.8s per sample) and outperforms same-scale VLMs by $+63.83$pp on VSP. A Gaussian-noise control ($+0.40$pp vs. $+10.28$pp for real VTs) and $58.4\%$ closure of the KL gap confirm that gains come from the semantic content of the generation pathway.

Problem

• unified multimodal models
• visual thoughts
• inference efficiency
• multimodal reasoning
• reasoning distillation

Innovation

• On-Policy Self-Distillation
• Visual Thoughts
• Multimodal Reasoning
• Token-level Distillation
• Unified Multimodal Models