Making Sense of Scams: Understanding Scam Conversations Through Multi-Level Alignment

2026-04-26

AI Summary

Existing fraud detection systems rely on static signals and interruptive warnings, which struggle to capture the dynamic risk inherent in conversational interactions and lack effective non-intrusive interaction designs. This work proposes a novel fraud detection approach based on a multi-level alignment model, introducing— for the first time—the degree of alignment across lexical, syntactic, semantic, and contextual dimensions as dynamic risk indicators. The method reveals, with fine-grained precision, that alignment levels in fraudulent conversations significantly decline as the dialogue progresses. Building on this insight, the authors design a non-intrusive visual prompting mechanism and validate its efficacy through user studies: compared to a no-prompt baseline, the approach improves F1 score by 0.21 and substantially outperforms conventional keyword-triggered alerts.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Online scams often unfold gradually through interaction, yet existing detection systems predominantly rely on snapshot-based signals and interruptive warnings, revealing two research gaps in the lack of signals that represent scam risk within conversational dynamics and the underexplored design of non-interruptive interaction. To address these gaps, we introduce multi-level alignment-based hints, informed by the Interactive Alignment Model, as a new detection signal for supporting sensemaking in scam-related conversations. These hints operationalize low-level lexical and syntactic alignments and high-level semantic and situation-model alignments between conversational participants, making conversational dynamics visible to users. We first conduct a preliminary evaluation on real-life scam dialogues, showing that as conversations approach scam attempts, low-level alignment scores remain stable while high-level alignment scores systematically decline, revealing a consistent cross-level pattern indicative of scam progression. Building on this insight, we conduct a user study with thirty participants, indicating that relative to the no-hint baseline, multi-level alignment-based hints increase precision by 0.25, recall by 0.16, and F1 score by 0.21, yielding substantially larger gains than the marginal improvements achieved by keyword-triggered alerts. Statistical analyses reveal that the proposed hints support earlier and more stable confidence formation over time, with ablation results further highlighting the effectiveness of combining alignment hints across levels in achieving these advantages.

Problem

• online scams
• conversational dynamics
• detection signals
• non-interruptive interaction
• scam detection

Innovation

• multi-level alignment
• scam detection
• conversational dynamics
• non-interruptive hints
• interactive alignment model