"It Warned Me Just at the Right Moment": Exploring LLM-based Real-time Detection of Phone Scams

2025-02-06

AI Summary

Telephone fraud remains highly prevalent, yet conventional detection methods struggle to balance real-time responsiveness with high accuracy. This paper proposes the first LLM-driven fraud detection framework designed specifically for real-time telephony streams, pioneering the integration of large language models into ultra-low-latency voice dialogue scenarios. Our method introduces a dynamic fraud-intent assessment mechanism coupled with optimized alert-timing scheduling, jointly leveraging streaming dialogue modeling, fine-grained intent classification, and latency-sensitive inference orchestration. Under an end-to-end latency constraint of <500 ms, it achieves an F1-score of 89.2%, significantly enhancing early fraud identification. Key contributions include: (i) the first reliable deployment of LLMs under millisecond-level response requirements; (ii) empirical validation of the feasibility of jointly optimizing high recall and ultra-low latency; and (iii) a scalable technical paradigm for real-time anti-fraud systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Despite living in the era of the internet, phone-based scams remain one of the most prevalent forms of scams. These scams aim to exploit victims for financial gain, causing both monetary losses and psychological distress. While governments, industries, and academia have actively introduced various countermeasures, scammers also continue to evolve their tactics, making phone scams a persistent threat. To combat these increasingly sophisticated scams, detection technologies must also advance. In this work, we propose a framework for modeling scam calls and introduce an LLM-based real-time detection approach, which assesses fraudulent intent in conversations, further providing immediate warnings to users to mitigate harm. Through experiments, we evaluate the method's performance and analyze key factors influencing its effectiveness. This analysis enables us to refine the method to improve precision while exploring the trade-off between recall and timeliness, paving the way for future directions in this critical area of research.

Problem

• Real-time phone scam detection
• LLM-based fraud intent assessment
• Immediate user warning system

Innovation

• LLM-based real-time detection
• Assessing fraudulent intent
• Immediate user warnings