Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation

2026-09-16

AI Summary

研究解决了AI生成放射报告评价中因报告实践差异导致的问题，通过引入ReRef方法调整参考报告以保持临床解释一致性，从而提高评价准确性。

Abstract

Radiologists follow heterogeneous reporting practices. Two radiologists examining the same image and identifying the same clinical findings might nevertheless compose superficially distinct reports, varying in terminology, shorthand, formatting, and level of detail. These variations in reporting norms represent an under-appreciated obstacle in efforts to evaluate AI-based radiology report generation (RRG) models, where machine-generated reports are typically assessed based on their concordance with human-generated references. In this paper, we quantify the sensitivity of established evaluation metrics to variations in reporting practices, revealing impacts large enough to alter the rankings of models. We introduce a radiologist-informed taxonomy of variations in radiology reporting practice and a method (ReRef) that rewrites reference reports along the axes of our taxonomy while preserving clinical interpretation. For instance, when comparing the performance of nine RRG models on MIMIC-CXR using RadCliQ-v1, condensing the discussion of normal findings in the reference reports causes Libra to drop from first to second place while CheXOne rises from third to first. Our results suggest that many current metrics fail to decouple clinical interpretation from conformity to reporting practices and that choosing the ``right'' references that accurately reflect the desired reporting practices can be important in practice. To support future research, we release MIMIC-CXR-Ext-ReRef, a radiologist-validated dataset of 120 (original, alternative) reference report pairs derived from MIMIC-CXR.

Problem

• Radiology Report Generation
• Reporting Practices
• Evaluation Metrics
• Clinical Interpretation
• Reference Reports

Innovation

• ReRef
• reporting practice
• clinical interpretation
• MIMIC-CXR-Ext-ReRef