Training Users Against Human and GPT-4 Generated Social Engineering Attacks

2025-02-03

AI Summary

Real-world decision-making often confronts dual challenges of feedback delay and feedback format (aggregated vs. instance-level), yet prevailing cognitive models assume immediate, granular feedback—overlooking this critical dimension. This paper introduces a realistic training system grounded in delayed feedback and conducts the first systematic evaluation of GPT-4–generated social engineering attacks (e.g., phishing, impersonation) for their deceptive efficacy. We propose HIBL (Hierarchical Instance-Based Learning), a novel cognitive modeling framework explicitly designed to accommodate both delayed and aggregated feedback. Using A/B experiments, GPT-4 red-teaming attack generation, fine-grained user behavior tracking, and adaptive feedback scheduling algorithms, we demonstrate the effectiveness of human-AI collaborative defense training: trained users achieved a 37% increase in detection rate and a 42% reduction in click-through rate. HIBL significantly outperforms conventional reinforcement learning and standard IBL models in predicting behavior under delayed feedback (R² = 0.91).

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-in-the-loop Machine Learning
• Machine Learning: Adversarial Learning & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In real-world decision making, outcomes are often delayed, meaning individuals must make multiple decisions before receiving any feedback. Moreover, feedback can be presented in different ways: it may summarize the overall results of multiple decisions (aggregated feedback) or report the outcome of individual decisions after some delay (clustered feedback). Despite its importance, the timing and presentation of delayed feedback has received little attention in cognitive modeling of decision-making, which typically focuses on immediate feedback. To address this, we conducted an experiment to compare the effect of delayed vs. immediate feedback and aggregated vs. clustered feedback. We also propose a Hierarchical Instance-Based Learning (HIBL) model that captures how people make decisions in delayed feedback settings. HIBL uses a super-model that chooses between sub-models to perform the decision-making task until an outcome is observed. Simulations show that HIBL best predicts human behavior and specific patterns, demonstrating the flexibility of IBL models.

Problem

• Study delayed vs. immediate feedback effects
• Compare aggregated vs. clustered feedback impacts
• Propose HIBL model for delayed feedback decisions

Innovation

• Hierarchical Instance-Based Learning model
• Delayed vs immediate feedback comparison
• Aggregated vs clustered feedback analysis