Beyond Detection: Evaluating Defensive LLMs Against AI-Generated Social Engineering in Live Turn-by-Turn Interaction

2026-08-10

AI Summary

This study addresses the challenge that existing large language model (LLM) defense systems struggle to accurately identify structurally grounded social engineering attacks in real-time interactions, often relying solely on superficial cues. The authors construct an online rental dialogue corpus comprising 300 conversations spanning 20 scenarios, four structural failure modes, and three surface-level conditions, evaluating five defense models under both real-time turn-by-turn and static single-turn settings. They introduce a novel “Chain-of-Trust Localization” framework to assess defensive efficacy across four dimensions: intervention behavior, timing, structural risk localization, and false positive rate. Results reveal wide variability in model intervention rates (0%–96.3%), a common decoupling between correct interventions and accurate structural localization, particular difficulty in detecting asset-control-related failures, and significant disparities in model sensitivity to surface features.

Abstract

Generative AI makes social-engineering attacks more fluent, adaptive, and scalable, increasing the need for LLM-based de- fenders that can protect users during ongoing interactions. We ask whether such defenders identify the structural source of risk or merely react to surface cues. We formalize trust-chain localization: identifying whether an interaction fails at actor authority, asset control, verification sufficiency, or transaction path. We construct a controlled 300-case online-housing corpus spanning 20 scenario families, legitimate cases, four structural failure modes, and three surface conditions. Five defender models are evaluated on the same corpus in state- ful turn-by-turn and one-shot static settings, yielding 1,500 model-case evaluations per protocol and 3,000 in total. No model produced explicit unsafe compliance, yet defensive effectiveness varied sharply: intervention rates ranged from 0% to 96.3%. Protective action and correct structural localization were frequently decoupled, with models sometimes intervening while identifying the wrong trust component or recognizing a structural failure without taking protective action. Asset-control failures were a major localization bottleneck, surface sensitivity varied across models, and live-static differences were model-dependent. These findings show that safe-looking behavior alone is insufficient; live scam resistance must separately measure intervention, timing, structural localization, and false-positive behavior.

Problem

• social engineering
• trust-chain localization
• defensive LLMs
• generative AI
• live interaction

Innovation

• trust-chain localization
• social engineering defense
• live turn-by-turn interaction
• structural failure modes
• LLM-based defenders