BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Features

2026-02-17

AI Summary

This work addresses the challenges in brain tumor radiology report generation, where scarce open-source paired image-text data, hallucination-prone existing methods, and limited interpretability hinder clinical utility. The authors propose a novel two-stage decoupled framework that first extracts deterministic imaging features and then leverages a large language model to generate structured narrative reports—marking the first such decoupling in the brain tumor domain. This approach significantly enhances clinical consistency and interpretability of generated reports. The study also introduces BTReport-BraTS, the first synthetic brain tumor radiology report dataset. Generated reports closely align with clinical references, and the extracted imaging features demonstrate strong predictive performance for patient overall survival and IDH mutation status, substantially outperforming current baselines.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Recent advances in radiology report generation (RRG) have been driven by large paired image-text datasets; however, progress in neuro-oncology has been limited due to a lack of open paired image-report datasets. Here, we introduce BTReport, an open-source framework for brain tumor RRG that constructs natural language radiology reports using deterministically extracted imaging features. Unlike existing approaches that rely on large general-purpose or fine-tuned vision-language models for both image interpretation and report composition, BTReport performs deterministic feature extraction for image analysis and uses large language models only for syntactic structuring and narrative formatting. By separating RRG into a deterministic feature extraction step and a report generation step, the generated reports are completely interpretable and less prone to hallucinations. We show that the features used for report generation are predictive of key clinical outcomes, including survival and IDH mutation status, and reports generated by BTReport are more closely aligned with reference clinical reports than existing baselines for RRG. Finally, we introduce BTReport-BraTS, a companion dataset that augments BraTS imaging with synthetically generated radiology reports produced with BTReport. Code for this project can be found at https://github.com/KurtLabUW/BTReport.

Problem

• radiology report generation
• brain tumor
• paired image-report datasets
• neuro-oncology
• clinical interpretability

Innovation

• deterministic feature extraction
• radiology report generation
• brain tumor
• hallucination mitigation
• clinically relevant features