Poster: Exploring the Limits of Audio-Based Detection of Turkish Phone Call Scams

2026-06-23

AI Summary

This study addresses the scarcity of effective phone scam detection methods and annotated data for low-resource languages like Turkish by introducing the first publicly available multimodal Turkish phone scam dataset, comprising 100 aligned audio–text pairs. The authors systematically evaluate seven large language models—including Gemini 2.5, GPT-4o, and the Qwen series—across three input modalities: raw audio, automatically transcribed text, and human-corrected transcripts. Results demonstrate that text-based inputs significantly outperform direct audio processing, with automatic transcripts achieving performance comparable to human-corrected ones, thereby validating the efficacy of a text-centric approach in low-resource settings. This work represents the first focused effort on Turkish phone scam detection and fills a critical gap in multimodal resources and benchmark evaluations for this domain.

Abstract

Scam phone calls exploit vulnerable communities worldwide, yet research on detection has focused almost exclusively on English and other high-resource languages. In low-resource settings such as Turkish, detection is especially difficult, as annotated data is scarce and technological defenses remain limited. This research investigates how large language models (LLMs) can support scam detection in Turkish by introducing the first public multi-modal dataset of 100 aligned audio-transcript pairs of scam and benign conversations. We evaluate seven LLMs spanning three model families: Gemini 2.5 (Flash, Flash-Lite, Pro), GPT-4o, and Qwen (Max, Plus, Turbo), under three input conditions: raw audio, automatic speech-to-text transcripts, and transcripts refined by a native speaker. Our results suggest that transcript-based inputs consistently outperform direct audio processing, while human-corrected and uncorrected transcripts perform comparably. By centering a low-resource language and real world threat, this work highlights the urgent need for culturally and linguistically inclusive AI safety research and more robust multi-modal systems for fraud prevention.

Problem

• phone call scams
• low-resource languages
• audio-based detection
• Turkish
• fraud prevention

Innovation

• low-resource language
• multi-modal dataset
• large language models
• scam detection
• audio-text alignment