From Rookie to Expert: Manipulating LLMs for Automated Vulnerability Exploitation in Enterprise Software

2025-12-27

AI Summary

This work challenges the long-standing security assumption that exploit development requires specialized expertise, revealing how the widespread adoption of large language models (LLMs) undermines foundational software security principles. Specifically, it identifies a novel threat: non-technical users can leverage social engineering to manipulate LLMs into autonomously generating functional exploit code. To realize this threat, the authors propose RSA—a prompt-engineering framework comprising Role Assignment, Scenario Preset, and Action Guidance—that bypasses mainstream LLM safety mechanisms without jailbreaking or fine-tuning. RSA employs multi-round, CVE-driven prompting combined with social engineering modeling. Evaluated across five commercial LLMs—including GPT-4o, Gemini, and Claude—the framework achieves 100% success in generating executable exploits for multiple CVEs on the Odoo ERP platform, requiring only 3–4 interaction rounds on average. This fully automated approach significantly outperforms prior methods reliant on manual intervention.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

LLMs democratize software engineering by enabling non-programmers to create applications, but this same accessibility fundamentally undermines security assumptions that have guided software engineering for decades. We show in this work how publicly available LLMs can be socially engineered to transform novices into capable attackers, challenging the foundational principle that exploitation requires technical expertise. To that end, we propose RSA (Role-assignment, Scenario-pretexting, and Action-solicitation), a pretexting strategy that manipulates LLMs into generating functional exploits despite their safety mechanisms. Testing against Odoo -- a widely used ERP platform, we evaluated five mainstream LLMs (GPT-4o, Gemini, Claude, Microsoft Copilot, and DeepSeek) and achieved a 100% success rate: tested CVE yielded at least one working exploit within 3-4 prompting rounds. While prior work [13] found LLM-assisted attacks difficult and requiring manual effort, we demonstrate that this overhead can be eliminated entirely. Our findings invalidate core software engineering security principles: the distinction between technical and non-technical actors no longer provides valid threat models; technical complexity of vulnerability descriptions offers no protection when LLMs can abstract it away; and traditional security boundaries dissolve when the same tools that build software can be manipulated to break it. This represents a paradigm shift in software engineering -- we must redesign security practices for an era where exploitation requires only the ability to craft prompts, not understand code. Artifacts available at: https://anonymous.4open.science/r/From-Rookie-to-Attacker-D8B3.

Problem

• Demonstrates how LLMs enable non-experts to create software exploits
• Proposes RSA strategy to bypass LLM safety mechanisms for exploit generation
• Shows traditional security assumptions fail when tools build and break software

Innovation

• RSA strategy manipulates LLMs to bypass safety mechanisms
• Achieved 100% exploit success on Odoo with mainstream LLMs
• Eliminates manual effort by automating vulnerability exploitation via prompting