People are poorly equipped to detect AI-powered voice clones

2024-10-03

arXiv.org

AI Summary

This study reveals a critical vulnerability in human perception of AI-generated speech: participants misidentified cloned voices as authentic in 80% of identity-matching trials, and achieved only ~60% accuracy—significantly below chance-adjusted baseline—in binary AI-speech detection. To our knowledge, this is the first systematic quantification of voice cloning–induced identity confusion rates and detection performance. We employed auditory psychophysics paradigms with rigorous double-blind experimental design, tightly controlled speech stimuli, and standardized identity-matching and detection tasks. Results demonstrate that state-of-the-art AI speech synthesis has reached a level of perceptual fidelity that renders reliable human discrimination between genuine and synthetic speech practically infeasible. This poses severe risks to voice-based authentication systems and societal trust. Our work establishes a reproducible benchmark methodology for speech security evaluation and provides foundational empirical evidence for policy and technical mitigation strategies.

Technology Category

• Natural Language Processing: Speech
• Humans and AI: AI for Accessibility
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

As generative artificial intelligence (AI) continues its ballistic trajectory, everything from text to audio, image, and video generation continues to improve at mimicking human-generated content. Through a series of perceptual studies, we report on the realism of AI-generated voices in terms of identity matching and naturalness. We find human participants cannot consistently identify recordings of AI-generated voices. Specifically, participants perceived the identity of an AI-voice to be the same as its real counterpart approximately 80% of the time, and correctly identified a voice as AI generated only about 60% of the time.

Problem

• Voice Recognition
• Artificial Intelligence
• Human-AI Discrimination

Innovation

• AI voice synthesis
• realism quantification
• discernment challenges