BanglaFake: Constructing and Evaluating a Specialized Bengali Deepfake Audio Dataset

2025-05-16

AI Summary

Deepfake audio detection for low-resource languages like Bengali faces critical challenges—including severe data scarcity and weak acoustic discriminative features. Method: We construct the first large-scale, high-quality Bengali deepfake audio benchmark (25,520 samples, comprising authentic and synthetic speech), and introduce the first language-specific detection benchmark. Leveraging state-of-the-art TTS models, we generate highly naturalistic fake speech; then conduct MFCC-based feature extraction, t-SNE visualization, and native-speaker MOS evaluations to identify representational bottlenecks in MFCCs for detecting forgery cues. Results: Synthetic speech achieves Robust-MOS scores of 3.40 (naturalness) and 4.01 (intelligibility). Our dataset significantly improves downstream detector training efficacy and evaluation reliability, establishing a foundational resource for deepfake detection research in low-resource languages.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deepfake audio detection is challenging for low-resource languages like Bengali due to limited datasets and subtle acoustic features. To address this, we introduce BangalFake, a Bengali Deepfake Audio Dataset with 12,260 real and 13,260 deepfake utterances. Synthetic speech is generated using SOTA Text-to-Speech (TTS) models, ensuring high naturalness and quality. We evaluate the dataset through both qualitative and quantitative analyses. Mean Opinion Score (MOS) from 30 native speakers shows Robust-MOS of 3.40 (naturalness) and 4.01 (intelligibility). t-SNE visualization of MFCCs highlights real vs. fake differentiation challenges. This dataset serves as a crucial resource for advancing deepfake detection in Bengali, addressing the limitations of low-resource language research.

Problem

• Addressing deepfake audio detection challenges in low-resource Bengali language
• Introducing a specialized Bengali dataset with real and synthetic speech samples
• Evaluating dataset quality through naturalness and intelligibility metrics

Innovation

• Constructed specialized Bengali deepfake audio dataset
• Used SOTA Text-to-Speech models for synthesis
• Evaluated with MOS and t-SNE visualization