Anticipate, Simulate, Reason (ASR): A Comprehensive Generative AI Framework for Combating Messaging Scams

2025-07-23

AI Summary

Rising SMS fraud in instant messaging poses significant security and societal risks. Method: This paper proposes ASR, a generative AI–based anti-fraud framework, introducing the novel “anticipate–simulate–reason” three-stage paradigm. It develops ScamGPT-J, a domain-specific large language model fine-tuned on a high-quality, multi-scenario scam dialogue dataset to enable fine-grained scam behavior modeling and interpretable reasoning. Contribution/Results: Experiments demonstrate that ASR significantly improves fraud detection accuracy—especially in complex, low-signal scenarios such as job-scam detection—while maintaining interpretability. Notably, it uncovers a paradoxical phenomenon: users at highest fraud risk exhibit comparatively lower acceptance of AI-assisted interventions. This work advances explainable, human-centered AI for fraud prevention and establishes a new deployment paradigm for generative AI in proactive security interventions.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

The rapid growth of messaging scams creates an escalating challenge for user security and financial safety. In this paper, we present the Anticipate, Simulate, Reason (ASR) framework, a generative AI method that enables users to proactively identify and comprehend scams within instant messaging platforms. Using large language models, ASR predicts scammer responses, creates realistic scam conversations, and delivers real-time, interpretable support to end-users. We develop ScamGPT-J, a domain-specific language model fine-tuned on a new, high-quality dataset of scam conversations covering multiple scam types. Thorough experimental evaluation shows that the ASR framework substantially enhances scam detection, particularly in challenging contexts such as job scams, and uncovers important demographic patterns in user vulnerability and perceptions of AI-generated assistance. Our findings reveal a contradiction where those most at risk are often least receptive to AI support, emphasizing the importance of user-centered design in AI-driven fraud prevention. This work advances both the practical and theoretical foundations for interpretable, human-centered AI systems in combating evolving digital threats.

Problem

• Proactively identify scams in messaging platforms
• Enhance scam detection using generative AI
• Address user vulnerability to messaging scams

Innovation

• Generative AI framework for scam detection
• Fine-tuned ScamGPT-J model on scam data
• Real-time interpretable support for users