SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning

2026-05-09

AI Summary

This work investigates the consistency of reasoning capabilities and the effective utilization of visual information in multimodal physical reasoning models during text-to-image modality transfer. To this end, the authors introduce SeePhys Pro, the first fine-grained benchmark enabling progressive modality transfer analysis, which comprises four semantically aligned question variants with incrementally increasing visual complexity. Through diagnostic methods—including blind training, image masking, text ablation, and format saturation analysis—the study reveals a significant performance drop as visual information becomes more complex. While blind training improves validation accuracy, it primarily exploits residual textual and distributional cues rather than genuine visual reasoning, thereby exposing a critical bottleneck in current models’ ability to ground visual variables in physical reasoning tasks.

Abstract

We introduce SeePhys Pro, a fine-grained modality transfer benchmark that studies whether models preserve the same reasoning capability when critical information is progressively transferred from text to image. Unlike standard vision-essential benchmarks that evaluate a single input form, SeePhys Pro features four semantically aligned variants for each problem with progressively increasing visual elements. Our evaluation shows that current frontier models are far from representation-invariant reasoners: performance degrades on average as information moves from language to diagrams, with visual variable grounding as the most critical bottleneck. Motivated by this inference-time fragility, we further develop large training corpora for multimodal RLVR and use blind training as a diagnostic control, finding that RL with all training images masked can still improve performance on unmasked validation sets. To analyze this effect, text-deletion, image-mask-rate, and format-saturation controls suggest that such gains can arise from residual textual and distributional cues rather than valid visual evidence. Our results highlight the need to evaluate multimodal reasoning not only by final-answer accuracy, but also by robustness under modality transfer and by diagnostics that test whether improvements rely on task-critical visual evidence.

Problem

• modality transfer
• multimodal reasoning
• visual grounding
• blind training
• physics reasoning

Innovation

• modality transfer
• blind training
• multimodal reasoning
• visual grounding
• representation invariance