sFRC for assessing hallucinations in medical image restoration

2026-03-04

AI Summary

This work addresses the critical yet underexplored issue of hallucinations in deep learning–based medical image reconstruction, where outputs often appear visually realistic but contain content distortions. To tackle the lack of effective detection methods, the authors propose a scanning analysis framework based on subregion Fourier Ring Correlation (sFRC), introducing localized frequency-domain correlation analysis for the first time to medical hallucination detection. The approach supports both expert-annotated and imaging-theory–driven hallucination mapping and demonstrates broad applicability across diverse reconstruction tasks—including CT super-resolution, sparse-view CT, and undersampled MRI. In CT, it effectively identifies hallucinated regions; in MRI, its findings align closely with theoretically predicted hallucination patterns. Furthermore, the method quantifies hallucination prevalence under varying data distributions and undersampling rates, confirming its generalizability and practical utility.

Abstract

Deep learning (DL) methods are currently being explored to restore images from sparse-view-, limited-data-, and undersampled-based acquisitions in medical applications. Although outputs from DL may appear visually appealing based on likability/subjective criteria (such as less noise, smooth features), they may also suffer from hallucinations. This issue is further exacerbated by a lack of easy-to-use techniques and robust metrics for the identification of hallucinations in DL outputs. In this work, we propose performing Fourier Ring Correlation (FRC) analysis over small patches and concomitantly (s)canning across DL outputs and their reference counterparts to detect hallucinations (termed as sFRC). We describe the rationale behind sFRC and provide its mathematical formulation. The parameters essential to sFRC may be set using predefined hallucinated features annotated by subject matter experts or using imaging theory-based hallucination maps. We use sFRC to detect hallucinations for three undersampled medical imaging problems: CT super-resolution, CT sparse view, and MRI subsampled restoration. In the testing phase, we demonstrate sFRC's effectiveness in detecting hallucinated features for the CT problem and sFRC's agreement with imaging theory-based outputs on hallucinated feature maps for the MR problem. Finally, we quantify the hallucination rates of DL methods on in-distribution versus out-of-distribution data and under increasing subsampling rates to characterize the robustness of DL methods. Beyond DL-based methods, sFRC's effectiveness in detecting hallucinations for a conventional regularization-based restoration method and a state-of-the-art unrolled method is also shown.

Problem

• hallucinations
• medical image restoration
• deep learning
• Fourier Ring Correlation
• image quality assessment

Innovation

• sFRC
• hallucination detection
• medical image restoration
• Fourier Ring Correlation
• deep learning robustness