Benchmarking Deep Learning-Based Reconstruction Methods for Photoacoustic Computed Tomography with Clinically Relevant Synthetic Datasets.

2026-01-23

arXiv.org

AI Summary

This work addresses the lack of a standardized, clinically relevant evaluation framework for deep learning–based reconstruction methods in photoacoustic computed tomography (PACT), where conventional image quality metrics often fail to reflect lesion recovery fidelity. To bridge this gap, the authors propose a comprehensive benchmarking framework centered on PACT, introducing—for the first time—a large-scale synthetic breast dataset featuring anatomically plausible structures and clinically relevant lesions. The evaluation integrates traditional image quality metrics with task-oriented assessment strategies. Experimental results reveal that certain deep learning approaches, despite achieving high scores on conventional metrics, fail to accurately reconstruct lesions, thereby underscoring the critical need for task-driven evaluation. This framework establishes a reproducible, objective, and clinically trustworthy standard for assessing PACT reconstruction algorithms.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Search and Optimization: Evaluation and Analysis
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Deep learning (DL)-based image reconstruction methods for photoacoustic computed tomography (PACT) have developed rapidly in recent years. However, most existing methods have not employed standardized datasets, and their evaluations rely on traditional image quality (IQ) metrics that may lack clinical relevance. The absence of a standardized framework for clinically meaningful IQ assessment hinders fair comparison and raises concerns about the reproducibility and reliability of reported advancements in PACT. A benchmarking framework is proposed that provides open-source, anatomically plausible synthetic datasets and evaluation strategies for DL-based acoustic inversion methods in PACT. The datasets each include over 11,000 two-dimensional (2D) stochastic breast objects with clinically relevant lesions and paired measurements at varying modeling complexity. The evaluation strategies incorporate both traditional and task-based IQ measures to assess fidelity and clinical utility. A preliminary benchmarking study is conducted to demonstrate the framework's utility by comparing DL-based and physics-based reconstruction methods. The benchmarking study demonstrated that the proposed framework enabled comprehensive, quantitative comparisons of reconstruction performance and revealed important limitations in certain DL-based methods. Although they performed well according to traditional IQ measures, they often failed to accurately recover lesions. This highlights the inadequacy of traditional metrics and motivates the need for task-based assessments. The proposed benchmarking framework enables systematic comparisons of DL-based acoustic inversion methods for 2D PACT. By integrating clinically relevant synthetic datasets with rigorous evaluation protocols, it enables reproducible, objective assessments and facilitates method development and system optimization in PACT.

Problem

• photoacoustic computed tomography
• deep learning
• image reconstruction
• benchmarking
• clinical relevance

Innovation

• benchmarking framework
• synthetic dataset
• task-based evaluation
• photoacoustic computed tomography
• deep learning reconstruction