Evaluating AI Models' Capability to Automate Voice Phishing Attacks

2026-07-10

AI Summary

This study systematically evaluates the feasibility and public risk of AI-driven voice phishing attacks. Through a large-scale survey experiment combined with qualitative interviews, it assesses the susceptibility of U.S. adults to synthetic voice phishing content generated by mainstream AI models, including Llama FD, Gemini, and ElevenLabs. The research provides the first quantitative evidence that AI-generated voices achieve an overall deception success rate of 16.5% in realistic phishing scenarios, rising to 36% under specific conditions. Notably, certain AI models demonstrate persuasive efficacy comparable to—or slightly exceeding—that of human speakers, and such attacks are already economically viable. These findings underscore the tangible threat posed by automated voice-based scams and offer empirical grounding for policymaking and the development of defensive countermeasures.

Abstract

Voice phishing (vishing) attacks have traditionally been limited by the need for human operators. The rapid emergence of high-quality AI voice synthesis and large language models (LLMs) reduces this bottleneck and enables scalable, automated scams. In this paper, we conduct a large-scale survey experiment (N=4100) and qualitative interviews (N=12) to assess U.S. adults' susceptibility to AI-powered voice phishing attacks. Participants were exposed to audio recordings or transcripts of scam scenarios generated using leading voice models such as Llama Full Duplex (Llama FD), Sesame, Gemini, OAI AVM, Play.AI, and ElevenLabs and the corresponding human baselines. The results show high compliance rates. Up to 36% of participants would or might comply with phishing requests in the "relative-in-distress" category. Overall compliance rate across all five scam categories was 16.5%, a striking figure given the low cost and high scalability of AI-automated voice phishing. Caller persuasiveness was the strongest predictor of compliance and certain models (most notably Sesame) achieved ratings comparable to human voices, or sometimes even slightly surpassing them. Our economic analysis suggests that while human-operated vishing is unprofitable at US wages, AI-powered vishing appears to be economically viable for several models. The primary risk of present-day AI-enabled vishing thus lies in the economics of automation rather than novel or "superhuman" persuasive techniques, though these cannot be ruled out for future systems. This raises significant concerns for the design of AI systems, consumer protection, and model release policies.

Problem

• voice phishing
• AI voice synthesis
• large language models
• social engineering
• automated scams

Innovation

• AI-powered voice phishing
• large language models
• voice synthesis
• social engineering
• automation economics