Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data

2026-06-24

AI Summary

This work addresses the lack of spatial grounding in existing medical vision-language models for brain MRI diagnosis, which often leads to unverifiable hallucinations. The authors propose BrReMark, a framework that enables two-stage reasoning through explicit region marking: first generating hypotheses by predicting abnormalities and annotating bounding boxes, then verifying diagnoses based on these marked regions. The approach integrates structured supervised fine-tuning, reinforcement learning with a localization-diagnosis composite reward, and domain-randomized pathological synthesis for data augmentation. Evaluated on an internal benchmark, BrReMark improves mAP50 from 0.74% to 37.54%, achieves a clinical F1 score of 21.57%, and attains 45.26% diagnostic accuracy. On the NOVA out-of-distribution benchmark, it reduces false positives by 45.7%, demonstrating enhanced reliability and generalization.

Abstract

Medical vision-language models typically generate diagnoses through single-pass inference without indicating which image regions support their conclusions. This lack of spatial grounding limits clinical utility: outputs cannot be audited, and models may hallucinate findings on normal scans. We present BrReMark (Brain Rethink via ROI Marking), a framework that introduces explicit region marking into brain MRI diagnosis. The model first generates hypotheses about potential abnormalities and grounds them through explicit bounding box marking, then verifies conclusions by re-examining the marked evidence. Training combines supervised fine-tuning on structured reasoning trajectories with reinforcement learning using a composite reward over localization accuracy and diagnostic reasoning. Furthermore, we integrate a domain randomization-based pathology synthesis augmentation strategy to improve the model's generalizability to out-of-distribution (OOD) data. On internal benchmark, BrReMark improves mAP50 from 0.74% to 37.54% compared to the base model, while achieving 21.57% Clinical F1 and 45.26% diagnostic accuracy. On NOVA OOD benchmark, it also achieves competitive overall performance with a 45.7% reduction in false positives compared to the state-of-the-art, indicating reduced hallucination on rare pathologies. These findings suggest that explicit hypothesis-verification grounding is a practical path toward trustworthy open-ended brain MRI diagnosis across both in-distribution and OOD settings.

Problem

• brain MRI
• anomaly detection
• spatial grounding
• hallucination
• out-of-distribution

Innovation

• region-of-interest marking
• hypothesis-verification grounding
• synthetic pathology augmentation
• medical vision-language model
• out-of-distribution generalization