E-MRL: Cross-view Aligned Evidence-driven Multimodal Reinforcement Learning for Reliable 3D Tumor Analysis

2026-06-22

AI Summary

This work addresses the susceptibility of existing vision-language models to generate visually ungrounded hallucinations in 3D tumor analysis. The authors formulate diagnosis as a Markov decision process structured around “diagnosis–localization–verification,” leveraging multimodal reinforcement learning to jointly produce a global diagnostic report and corresponding key evidence slices. A cross-view consistency reward mechanism is introduced to enforce semantic alignment between local image slices and the global diagnostic conclusion, thereby enabling interpretable and visually grounded reasoning. Evaluated on a large-scale 3D CT tumor dataset, the proposed method significantly reduces hallucination rates and improves diagnostic accuracy, outperforming both supervised fine-tuning and conventional reinforcement learning baselines.

Abstract

While Vision-Language Models (VLMs) show great promise in volumetric medical report generation, they frequently suffer from visual hallucinations and a lack of grounding in 3D CT data. Current Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) strategies typically optimize text fidelity alone, essentially rewarding correct diagnoses derived from language priors rather than genuine visual perception. To address this, we propose cross-view aligned Evidence-driven Multimodal Reinforcement Learning (Evidence-MRL, noted as E-MRL), a reliable RL reasoning framework that formulates the generation process as a Markov Decision Process of "diagnosis-localization-verification". Unlike standard approaches, our model is explicitly trained to identify a "key evidence slice" alongside the global diagnostic report, grounding its findings in verifiable visual evidence. Crucially, we introduce a novel cross-view consistency reward, which validates the semantic alignment between the golden-standard report and a local visual re-query of the selected key slice, providing additional rewards for correctly-localized reasoning. Experiments on large-scale 3D CT tumor datasets demonstrate that E-MRL significantly reduces hallucinations and improves diagnostic accuracy compared to SFT and RL baselines, offering a clinically interpretable solution for visually-grounded and tumor analysis.

Problem

• visual hallucination
• 3D tumor analysis
• vision-language models
• evidence grounding
• medical report generation

Innovation

• Evidence-driven Reinforcement Learning
• Cross-view Alignment
• Key Evidence Slice
• Visual Grounding
• 3D Tumor Analysis