Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context

2026-07-07

AI Summary

This work addresses the growing imbalance between surging demand for medical imaging and limited radiologist capacity by proposing the first multimodal large language model capable of generating both structured and unstructured radiology reports that meet the diagnostic threshold of the FRCR 2B examination. The model integrates imaging data, prior examinations, and clinical text through a three-stage training pipeline: domain adaptation, curriculum learning–driven contrastive optimization of the visual encoder, and multi-turn visual question answering fine-tuning. A novel Findings-Diagnosis scoring framework based on ontological synonym matching and polarity contradiction detection is introduced to enhance clinical fidelity. Evaluated across RadBench, ReXGradient, and an internal multimodal dataset encompassing chest, skeletal, abdominal, spinal, pelvic radiographs, and mammograms, the model achieves state-of-the-art performance, significantly improving report accuracy and clinical utility.

Abstract

Imaging demand is growing faster than the radiology workforce can expand, and reporting backlogs cannot be resolved through training and recruitment alone. The most direct opportunity is reducing the time and effort radiologists spend producing reports, a task that requires interpreting images, integrating clinical history and prior studies, and drafting structured findings. We present Harrison.Rad 1.5 (HR1.5), a radiology-specific multimodal large language model that accepts interleaved text and visual inputs and generates structured and unstructured text across plain-film radiology, spanning computed radiography, chest, musculoskeletal, abdominal, spine, and pelvic x-rays, and mammography. HR1.5 is trained through a three-stage pipeline: domain adaptation of a base language model on radiology reports, contrastive vision-encoder training with curriculum-based hard negatives on ~6 million image-report instances, and visual-question-answering fine-tuning on multi-turn conversations. We evaluate it with a Findings-Diagnosis scoring framework that extends RadGraph-XL entity extraction with ontology-based synonym matching and polarity-contradiction detection, benchmarked on RadBench, a simulated FRCR 2B Short Case examination scored against Angoff-method thresholds, ReXGradient, and internal multi-modality datasets. HR1.5 is the only system evaluated to meet the simulated FRCR passing standard and achieves the highest accuracy on closed-format clinical questions, across anatomical regions, on internal multi-body-part and mammography reporting, and on the primary clinically-aligned score for public chest reporting. We further examine explainability and model behaviour, including question-sensitive Grad-CAM heatmaps, attention analysis, and confidence estimation, to support responsible future evaluation toward clinical use, and a framework for clinically grounded assessment of report quality.

Problem

• radiology reporting
• imaging demand
• reporting backlog
• radiologist workload
• clinical documentation

Innovation

• multimodal large language model
• radiology foundation model
• structured report generation
• contrastive vision-language training
• clinically grounded evaluation