Rule-ATT&CK Mapper (RAM): Mapping SIEM Rules to TTPs Using LLMs

2025-02-04

AI Summary

Manual mapping of SIEM rules to MITRE ATT&CK techniques (TTPs) is inefficient and error-prone, while existing machine learning approaches struggle to handle the structured nature of SIEM rules. Method: We propose the first multi-stage prompt-chaining LLM framework specifically designed for SIEM rule-to-TTP mapping—requiring no pretraining or fine-tuning. It leverages structured prompt engineering and injection of external cybersecurity knowledge to enhance domain-specific TTP identification capabilities of large language models (e.g., GPT-4-Turbo, Qwen, Granite, Mistral). Contribution/Results: Evaluated on the Splunk Security Content dataset, GPT-4-Turbo achieves the highest accuracy. Ablation studies confirm that external knowledge substantially compensates for LLMs’ deficiencies in implicit cybersecurity knowledge. This work establishes a new paradigm for automated, interpretable, and scalable annotation of threat-detection rules.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Rule Mining & Pattern Mining
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The growing frequency of cyberattacks has heightened the demand for accurate and efficient threat detection systems. SIEM platforms are important for analyzing log data and detecting adversarial activities through rule-based queries, also known as SIEM rules. The efficiency of the threat analysis process relies heavily on mapping these SIEM rules to the relevant attack techniques in the MITRE ATT&CK framework. Inaccurate annotation of SIEM rules can result in the misinterpretation of attacks, increasing the likelihood that threats will be overlooked. Existing solutions for annotating SIEM rules with MITRE ATT&CK technique labels have notable limitations: manual annotation of SIEM rules is both time-consuming and prone to errors, and ML-based approaches mainly focus on annotating unstructured free text sources rather than structured data like SIEM rules. Structured data often contains limited information, further complicating the annotation process and making it a challenging task. To address these challenges, we propose Rule-ATT&CK Mapper (RAM), a novel framework that leverages LLMs to automate the mapping of structured SIEM rules to MITRE ATT&CK techniques. RAM's multi-stage pipeline, which was inspired by the prompt chaining technique, enhances mapping accuracy without requiring LLM pre-training or fine-tuning. Using the Splunk Security Content dataset, we evaluate RAM's performance using several LLMs, including GPT-4-Turbo, Qwen, IBM Granite, and Mistral. Our evaluation highlights GPT-4-Turbo's superior performance, which derives from its enriched knowledge base, and an ablation study emphasizes the importance of external contextual knowledge in overcoming the limitations of LLMs' implicit knowledge for domain-specific tasks. These findings demonstrate RAM's potential in automating cybersecurity workflows and provide valuable insights for future advancements in this field.

Problem

• Automate mapping SIEM rules to MITRE ATT&CK techniques
• Enhance accuracy of cyber threat detection systems
• Leverage LLMs for structured data annotation

Innovation

• LLMs automate mapping of SIEM rules
• Multi-stage pipeline enhances mapping accuracy
• GPT-4-Turbo excels in cybersecurity tasks