Towards Detecting Neural Audio Codec Synthesized Heart Sounds

2026-06-19

AI Summary

This study addresses the emerging risk of physiological signal spoofing posed by neural audio codecs capable of synthesizing phonocardiogram (PCG) signals. To this end, it introduces the first synthetic heart sound detection task, termed SHAC, and presents CARDIOFAKE—the first benchmark dataset encompassing both genuine and synthetically generated heart sounds. The authors propose GROOT, a multimodal feature fusion framework that effectively integrates spectral features such as MFCCs and LFCCs with self-supervised representations from models like WavLM. Experimental results demonstrate that GROOT significantly outperforms existing baselines and single-feature approaches on the SHAC task, achieving state-of-the-art performance and thereby advancing research in this nascent domain.

Abstract

In this paper, we introduce Synthetic Heart Sound Detection (SHAC), a task aimed at identifying phonocardiograms (PCGs) synthesized using neural audio codecs (NACs). To facilitate research in this direction, we release CARDIOFAKE, the first benchmark dataset for SHAC containing both real and codec-synthesized PCGs. We benchmark spectral representations (MFCC, LFCC) and self-supervised learning (SSL) representations (e.g., WavLM) for the task. Furthermore, we propose GROOT, a fusion framework that integrates spectral and SSL features for leveraging their complementary behavior. Experiments show that GROOT, combining MFCC and WavLM, achieves state-of-the-art performance, outperforming individual representations and competitive baselines.

Problem

• Synthetic Heart Sound Detection
• Neural Audio Codec
• Phonocardiogram
• Audio Forgery Detection

Innovation

• Synthetic Heart Sound Detection
• Neural Audio Codec
• Self-Supervised Learning
• Feature Fusion
• CARDIOFAKE