🤖 AI Summary
This work addresses the growing risk of synthetic speech misuse by proposing a novel task—“audio avatar fingerprinting”—to verify whether synthesized utterances originate from an authorized identity. Building upon existing speaker verification models, the approach enables authentication of synthetic speech against legitimate user profiles. To support this task, we introduce the first forensic dataset comprising paired authentic and corresponding synthetic speech samples. Using this dataset, we demonstrate the feasibility of reliably distinguishing authorized synthetic speech from unauthorized or spoofed instances. Beyond defining a new research direction at the intersection of audio forensics and trustworthy synthetic media, this study provides a foundational benchmark dataset and evaluation framework to advance future work in secure and accountable text-to-speech applications.
📝 Abstract
With the advancements in AI speech synthesis, it is easier than ever before to generate realistic audio in a target voice. One only needs a few seconds of reference audio from the target, quite literally putting words in the target person's mouth. This imposes a new set of forensics-related challenges on speech-based authentication systems, videoconferencing, and audio-visual broadcasting platforms, where we want to detect synthetic speech. At the same time, leveraging AI speech synthesis can enhance the different modes of communication through features such as low-bandwidth communication and audio enhancements - leading to ever-increasing legitimate use-cases of synthetic audio. In this case, we want to verify if the synthesized voice is actually spoken by the user. This will require a mechanism to verify whether a given synthetic audio is driven by an authorized identity, or not. We term this task audio avatar fingerprinting. As a step towards audio forensics in these new and emerging situations, we analyze and extend an off-the-shelf speaker verification model developed outside of forensics context for the task of fake speech detection and audio avatar fingerprinting, the first experimentation of its kind. Furthermore, we observe that no existing dataset allows for the novel task of verifying the authorized use of synthetic audio - a limitation which we address by introducing a new speech forensics dataset for this novel task.