Fully Automated End-to-End Adversary Emulation from MITRE ATT\&CK Based Cyber Threat Intelligence Using LLMs

2026-07-16

AI Summary

This work proposes an end-to-end automated red-teaming framework that overcomes the limitations of existing adversary emulation approaches, which typically rely on predefined playbooks or manual intervention and struggle to automatically derive executable attack procedures from cyber threat intelligence (CTI). The proposed system uniquely integrates MITRE ATT&CK-aligned CTI report parsing, large language model–driven attack playbook generation, and an automatic execution pipeline with failure-type-aware repair mechanisms—all within a unified workflow requiring no human involvement. Implemented on the CALDERA platform and leveraging models such as Claude Sonnet 4.5, the framework generates an average of 27.3 attack capabilities per CTI report across 11 reports, achieving an 84.22% post-repair execution success rate and an F1 score of 60.50%, significantly outperforming AURORA.

Abstract

This paper presents a fully automated end-to-end framework for adversary emulation from MITRE ATT&CK-aligned CTI reports using LLMs. Unlike prior work, which either executes prewritten playbooks or partially automates playbook generation, our framework unifies playbook generation, execution, and failure recovery in a single workflow. In particular, although AURORA, the most recent prior study, generates playbooks from CTI reports, it still requires partial manual intervention and does not revise playbooks based on execution failures. Our framework generates Caldera playbooks from CTI reports, executes them automatically, and revises failed Abilities through a failure-type-aware recovery mechanism. Evaluated on 11 CTI reports with Claude Sonnet 4.5, GPT-4o, Gemini 2.5 Pro, and Grok 4 Fast, the framework achieved its best results with Claude Sonnet 4.5: 27.3 Abilities per playbook, 84.22% execution success after revision, and CTI Precision, Recall, and F1 of 73.95%, 52.48%, and 60.50%, respectively. The failure recovery mechanism consistently improved execution success across all evaluated LLM models by 14.59%p to 17.23%p. On the 10 CTI reports selected from AURORA's dataset, this mechanism further increased the final execution success rate, surpassing that of AURORA, which represents the state-of-the-art adversary emulation system.

Problem

• adversary emulation
• MITRE ATT&CK
• cyber threat intelligence
• playbook automation
• failure recovery

Innovation

• adversary emulation
• LLM-based automation
• failure-aware recovery
• MITRE ATT&CK
• end-to-end playbook generation