The Impact of Emerging Phishing Threats: Assessing Quishing and LLM-generated Phishing Emails against Organizations

2025-05-17

AI Summary

Evolving phishing threats—particularly QR code–based “quishing” and large language model (LLM)–generated spear-phishing emails—lack empirical validation regarding their real-world efficacy and the effectiveness of current defenses. Method: We conducted three large-scale, realistic organizational phishing simulations across 71,000+ employees, integrating QR code engineering, LLM-driven social engineering content generation, open-source intelligence, and validated security awareness questionnaires. Contribution/Results: Quishing achieved click-through rates comparable to traditional phishing but exhibited significantly lower detection rates; LLM-crafted emails drove >30% landing-page visitation post-open; and self-reported security awareness robustly predicted individual phishing resilience. This study provides the first systematic, evidence-based assessment of these emerging attack vectors and delivers actionable insights for strengthening human-centric cybersecurity defenses.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Security and Privacy: Large-scale security measurements

Abstract

Modern organizations are persistently targeted by phishing emails. Despite advances in detection systems and widespread employee training, attackers continue to innovate, posing ongoing threats. Two emerging vectors stand out in the current landscape: QR-code baits and LLM-enabled pretexting. Yet, little is known about the effectiveness of current defenses against these attacks, particularly when it comes to real-world impact on employees. This gap leaves uncertainty around to what extent related countermeasures are justified or needed. Our work addresses this issue. We conduct three phishing simulations across organizations of varying sizes -- from small-medium businesses to a multinational enterprise. In total, we send over 71k emails targeting employees, including: a"traditional"phishing email with a click-through button; a nearly-identical"quishing"email with a QR code instead; and a phishing email written with the assistance of an LLM and open-source intelligence. Our results show that quishing emails have the same effectiveness as traditional phishing emails at luring users to the landing webpage -- which is worrying, given that quishing emails are much harder to identify even by operational detectors. We also find that LLMs can be very good"social engineers": in one company, over 30% of the emails opened led to visiting the landing webpage -- a rate exceeding some prior benchmarks. Finally, we complement our study by conducting a survey across the organizations' employees, measuring their"perceived"phishing awareness. Our findings suggest a correlation between higher self-reported awareness and organizational resilience to phishing attempts.

Problem

• Assessing effectiveness of defenses against QR-code phishing (quishing) attacks
• Evaluating impact of LLM-generated phishing emails on employee vulnerability
• Measuring correlation between perceived awareness and organizational phishing resilience

Innovation

• Conduct phishing simulations with QR-code baits
• Use LLM-generated pretexting in phishing emails
• Measure employee awareness via organizational surveys