Standardized Evaluation of Fetal Phonocardiography Processing Methods

2025-07-14

AI Summary

To address the lack of standardized evaluation protocols for fetal phonocardiogram (fPCG) signal detection and heart rate estimation, this paper introduces the first unified benchmarking framework. It proposes a composite evaluation scheme comprising tolerance-based detection accuracy, label editing error rate (insertion/deletion/substitution), and heart rate estimation root-mean-square error (RMSE). We establish an open-source benchmark platform and a harmonized multi-center dataset to ensure algorithm reproducibility and fair comparison. Our key innovation lies in incorporating physiological constraints—such as S1–S2 temporal ordering—into metric design, and we publicly release all evaluation code and baseline models. Experimental results demonstrate state-of-the-art performance: 97.6% F1-score for S1 detection, 91.4% F1-score for S2 detection, and an RMSE of only 0.644 bpm for heart rate estimation. This framework significantly enhances cross-study comparability and advances clinical standardization of fPCG analysis.

Technology Category

• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Motivation. Phonocardiography can give access to the fetal heart rate as well as direct heart sound data, and is entirely passive, using no radiation of any kind. Approach. We discuss the currently available methods for fetal heart sound detection and heart rate estimation and compare them using a common benchmarking platform and a pre-selected testing dataset. Compared to previous reviews, we evaluated the discussed methods in a standardized manner for a fair comparison. Our tests included tolerance-based detection accuracy, error rates for label insertions, deletions, and substitutions, and statistical measures for heart rate mean square error. Results. Based on our results, there is no definite best method that can achieve the highest scores in all of the tests, and simpler methods could perform comparably to more complex ones. The best model for first heart sound detection achieved 97.6% F1-score, 97.4% positive predictive value, and 12.2+-8.0 ms mean absolute error. In terms of second heart sound detection the best model had 91.4% F1-score, 91.3% positive predictive value, and 17.3+-12.2 ms mean absolute error. For fetal heart rate a 0.644 mean square error was achieved by the best method. Significance. Our main conclusion is that further standardization is required in fetal heart rate and heart sound detection method evaluation. The tests and algorithm implementations are openly available at: https://github.com/mulkr/standard-fpcg-evaluation.

Problem

• Standardized comparison of fetal heart sound detection methods
• Evaluation of fetal heart rate estimation accuracy
• Need for further standardization in fetal phonocardiography

Innovation

• Standardized benchmarking platform for fetal phonocardiography
• Comparative evaluation of heart sound detection methods
• Open-source algorithm implementations for fair testing