HoliAntiSpoof: Audio LLM for Holistic Speech Anti-Spoofing

2026-02-04

AI Summary

This work addresses the limitations of existing voice anti-spoofing approaches, which predominantly rely on binary classification and struggle to model the coupled effects of diverse spoofing techniques on both acoustic attributes and semantic content. To overcome this, we propose the first unified anti-spoofing framework based on Audio Large Language Models (ALLMs), reframing the task as a text generation problem to jointly infer the spoofing method, the affected speech attributes, and the resulting semantic distortions. We introduce DailyTalkEdit, a new benchmark enabling semantic-level evaluation, and incorporate in-context learning to enhance out-of-domain generalization. Experimental results demonstrate that our approach significantly outperforms conventional baselines across multiple settings while providing interpretable analyses of spoofing behaviors.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Speech
• Application Domains: Misinformation & Fake News

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advances in speech synthesis and editing have made speech spoofing increasingly challenging. However, most existing methods treat spoofing as binary classification, overlooking that diverse spoofing techniques manipulate multiple, coupled speech attributes and their semantic effects. In this paper, we introduce HoliAntiSpoof, the first audio large language model (ALLM) framework for holistic speech anti-spoofing analysis. HoliAntiSpoof reformulates spoofing analysis as a unified text generation task, enabling joint reasoning over spoofing methods, affected speech attributes, and their semantic impacts. To support semantic-level analysis, we introduce DailyTalkEdit, a new anti-spoofing benchmark that simulates realistic conversational manipulations and provides annotations of semantic influence. Extensive experiments demonstrate that HoliAntiSpoof outperforms conventional baselines across multiple settings, while preliminary results show that in-context learning further improves out-of-domain generalization. These findings indicate that ALLMs not only enhance speech spoofing detection performance but also enable interpretable analysis of spoofing behaviors and their semantic effects, pointing towards more trustworthy and explainable speech security. Data and code are publicly available.

Problem

• speech anti-spoofing
• audio large language model
• semantic impact
• speech attributes
• spoofing detection

Innovation

• Audio Large Language Model
• Holistic Anti-Spoofing
• Text Generation for Spoofing Analysis
• Semantic Impact Reasoning
• DailyTalkEdit Benchmark