When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs

2024-10-18

arXiv.org

AI Summary

This study systematically evaluates the practical threat posed by web-augmented large language model (LLM) agents in cybersecurity attacks, focusing on three high-risk scenarios: personally identifiable information (PII) exfiltration, identity impersonation via social media posting, and targeted phishing email generation. We implement LLM-based agents leveraging state-of-the-art foundation models and tool-augmented frameworks—integrated with search engines, social media APIs, and email templating modules—and conduct empirical evaluation through human assessment and controlled A/B testing. Our work provides the first quantitative evidence of LLM agents’ operational efficacy in real-world attacks: 95.9% PII extraction accuracy, 93.9% perceived authenticity of impersonated posts, and a 46.67% increase in phishing link click-through rates. Critically, we demonstrate widespread failure of current commercial LLM safety mitigations against such adversarial automation. These findings underscore how LLM agents substantially lower both technical barriers and economic costs of cyberattacks, delivering critical empirical grounding for AI security policy and governance.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Recent advancements in Large Language Models (LLMs) have established them as agentic systems capable of planning and interacting with various tools. These LLM agents are often paired with web-based tools, enabling access to diverse sources and real-time information. Although these advancements offer significant benefits across various applications, they also increase the risk of malicious use, particularly in cyberattacks involving personal information. In this work, we investigate the risks associated with misuse of LLM agents in cyberattacks involving personal data. Specifically, we aim to understand: 1) how potent LLM agents can be when directed to conduct cyberattacks, 2) how cyberattacks are enhanced by web-based tools, and 3) how affordable and easy it becomes to launch cyberattacks using LLM agents. We examine three attack scenarios: the collection of Personally Identifiable Information (PII), the generation of impersonation posts, and the creation of spear-phishing emails. Our experiments reveal the effectiveness of LLM agents in these attacks: LLM agents achieved a precision of up to 95.9% in collecting PII, generated impersonation posts where 93.9% of them were deemed authentic, and boosted click rate of phishing links in spear phishing emails by 46.67%. Additionally, our findings underscore the limitations of existing safeguards in contemporary commercial LLMs, emphasizing the urgent need for robust security measures to prevent the misuse of LLM agents.

Problem

• Large Language Models
• Cybersecurity Risks
• Internet Connectivity

Innovation

• Large Language Models
• Internet-Linked Risks
• Cybersecurity Challenges