Large Audio Language Models for Spoofing-Aware Speaker Verification

2026-07-16

AI Summary

This work addresses the serious spoofing threat posed by high-fidelity speech synthesis and voice cloning to automatic speaker verification systems. It presents the first systematic exploration of large audio language models (LALMs) for spoofing-aware speaker verification (SASV), introducing a unified, auditable framework that integrates zero-shot prompting, supervised fine-tuning, inference-oriented training, and reinforcement learning optimization. Experimental results demonstrate that while pretrained LALMs perform near-randomly in zero-shot settings, they achieve substantial performance gains after task-specific adaptation, matching the SASV accuracy of conventional cascaded systems. Moreover, the proposed approach enables natural language reasoning, thereby enhancing model interpretability and robustness against spoofing attacks.

Abstract

Recent advances in text-to-speech and voice cloning make high-quality spoofing inexpensive and scalable, threatening voice authentication systems, especially automatic speaker verification (ASV). Existing defenses mainly address this threat through binary countermeasures (CMs) for deepfake detection or spoofing-aware speaker verification (SASV), where current systems are dominated by modular ASV-CM fusion and cascaded pipelines. Although large audio language models (LALMs) have shown promise on related audio tasks, including CM and ASV, their use for SASV remains unexplored, despite their capacity to produce natural-language rationales for auditing and robustness beyond discriminative predictions. This work systematically evaluates LALMs for SASV against conventional pipelines under zero-shot prompting, supervised adaptation, reasoning-oriented training, and reinforcement-learning-based optimization. Our results show that pretrained LALMs are near chance in the zero-shot setting, confirming that they are not natively suited to SASV, but that task-specific adaptation closes this gap. We further find that competitive SASV performance can be achieved through several distinct routes. These findings position LALMs as a promising and auditable foundation for unified SASV, while clarifying where conventional cascade systems still lead.

Problem

• speaker verification
• spoofing
• deepfake detection
• audio language models
• voice authentication

Innovation

• Large Audio Language Models
• Spoofing-Aware Speaker Verification
• Zero-shot Prompting
• Reasoning-oriented Training
• Auditable AI