Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages

2026-06-14

AI Summary

This study addresses the limited generalization of existing voice deepfake detection methods on Southeast Asian (SEA) languages and the absence of a multilingual benchmark for neural audio codec–generated synthetic speech (Codecfakes). To bridge this gap, we introduce SEA-CF, the first large-scale Codecfakes detection benchmark tailored to SEA languages, encompassing diverse languages, speakers, and neural audio codecs. We also propose GARUDA, a lightweight audio language model specifically designed for low-resource and low-latency deployment scenarios. Experimental results demonstrate that state-of-the-art English-centric detectors suffer significant performance degradation on SEA languages, whereas GARUDA outperforms both end-to-end and large audio language model baselines on SEA-CF, achieving high accuracy while maintaining practical deployability.

Abstract

Codecfakes (CFs) are a type of speech deepfakes generated through Audio Language Models (ALMs), with Neural Audio Codecs (NACs) forming the core mechanism for speech encoding and generation. CFs exhibit distributional characteristics that differ from vocoder-based deepfakes, causing detectors trained on vocoder data to generalize poorly to CFs detection. Although this has led to the development of CF detection benchmarks, existing resources are largely confined to English -- and to a limited extent Chinese -- leaving South-East Asian (SEA) languages unexplored. To bridge this gap, we introduce SEA-CF, the first large-scale benchmark for CF detection spanning multiple SEA languages, diverse speaker profiles, and a wide range of NAC architectures. SEA-CF is constructed by synthesizing publicly available real speech corpora. Our experiments show that state-of-the-art (SOTA) CF detectors trained on English-centric datasets fail to generalize to SEA speech due to language-specific phonetic structures, tonal variations, and rich prosodic diversity. We further conduct a comprehensive zero-shot and fine-tuned evaluation of recent SOTA ALMs on SEA-CF. Fine-tuning the ALMs improves performance, however, these are very large being impractical for real-world application due to their scale, particularly in low-resource and latency-constrained settings. To address this limitation, we propose a novel small-ALM, GARUDA tailored for CF detection, which delivers strong performance while remaining lightweight. Extensive evaluations demonstrate that the proposed Small-ALM outperforms strong end-to-end and ALM-based baselines, establishing a new, practical direction for robust CF detection in SEA languages and beyond.

Problem

• Codecfakes
• Southeast Asian languages
• neural audio codecs
• speech deepfakes
• detection generalization

Innovation

• Codecfakes
• Neural Audio Codec
• SEA languages
• lightweight ALM
• deepfake detection