MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models
This work addresses the susceptibility of current vision-language models to visual perception errors and hallucinations during complex reasoning, a challenge exacerbated by the low sampling efficiency and sparse rewards of conventional reinforcement learning approaches that struggle to disentangle error sources. To overcome these limitations, the authors propose MIRL, a novel framework that introduces mutual information into vision-language reinforcement learning for the first time. Specifically, MIRL leverages the mutual information between generated captions and visual inputs as a pre-screening signal, employs a trajectory-forking mechanism to intelligently allocate sampling budgets, and adopts a decoupled training strategy that provides dedicated rewards for the visual perception stage. Evaluated across six benchmarks, MIRL achieves an average accuracy of 70.22%, surpassing the performance of methods using 16 full trajectories with only 10 pre-samples followed by top-6 selection—reducing sampling cost by 25% while significantly improving both training efficiency and accuracy.