Anatomical Similarity as a New Metric to Evaluate Brain Generative Models

2025-04-30

AI Summary

Current MRI synthesis evaluation overemphasizes texture and perceptual quality (e.g., PSNR, SSIM), neglecting clinically critical anatomical fidelity. To address this, we propose WASABI—a novel, differentiable, and comparable metric that quantifies anatomical plausibility via multivariate Wasserstein distance computed over regional brain volume distributions derived from SynthSeg-automated segmentation. Unlike conventional metrics, WASABI requires no manual annotations, is end-to-end differentiable, and effectively uncovers latent volumetric inaccuracies even in visually realistic synthetic images. Extensive experiments across two real-world MRI datasets and five state-of-the-art generative models demonstrate that WASABI significantly outperforms traditional metrics in sensitivity to anatomical distortions—particularly subtle volume deviations—and provides a more clinically meaningful assessment. By grounding evaluation in quantitative neuroanatomical consistency, WASABI shifts the paradigm of MRI synthesis validation from perception-oriented to clinical-trustworthiness-oriented.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data generation and simulation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Generative models enhance neuroimaging through data augmentation, quality improvement, and rare condition studies. Despite advances in realistic synthetic MRIs, evaluations focus on texture and perception, lacking sensitivity to crucial anatomical fidelity. This study proposes a new metric, called WASABI (Wasserstein-Based Anatomical Brain Index), to assess the anatomical realism of synthetic brain MRIs. WASABI leverages extit{SynthSeg}, a deep learning-based brain parcellation tool, to derive volumetric measures of brain regions in each MRI and uses the multivariate Wasserstein distance to compare distributions between real and synthetic anatomies. Based on controlled experiments on two real datasets and synthetic MRIs from five generative models, WASABI demonstrates higher sensitivity in quantifying anatomical discrepancies compared to traditional image-level metrics, even when synthetic images achieve near-perfect visual quality. Our findings advocate for shifting the evaluation paradigm beyond visual inspection and conventional metrics, emphasizing anatomical fidelity as a crucial benchmark for clinically meaningful brain MRI synthesis. Our code is available at https://github.com/BahramJafrasteh/wasabi-mri.

Problem

• Evaluating anatomical fidelity in synthetic brain MRIs
• Proposing WASABI metric for anatomical realism assessment
• Comparing real and synthetic brain MRI distributions

Innovation

• Proposes WASABI metric for anatomical realism
• Uses SynthSeg for brain region volumetrics
• Compares distributions with Wasserstein distance