On Automating Security Policies with Contemporary LLMs

2025-06-05

AI Summary

Addressing the challenge of automatically and accurately mapping high-level security policies to low-level API invocations in complex network environments, this paper proposes a security policy parsing framework that integrates context learning with retrieval-augmented generation (RAG). The framework synergistically leverages STIXv2 threat intelligence parsing, Windows API semantic modeling, and vector database indexing to enable dynamic adaptation across heterogeneous security tools and API specifications. Differing from prior approaches, this work is the first to systematically incorporate RAG into the security policy execution pipeline, significantly enhancing semantic alignment between policies and executable APIs. Experimental evaluation on the CTI policy dataset demonstrates that our method achieves average improvements exceeding 32% over baseline models in execution precision, recall, and F1-score—thereby validating its effectiveness and generalizability.

Technology Category

• Natural Language Processing: Safety and Robustness
• Application Domains: Security
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The complexity of modern computing environments and the growing sophistication of cyber threats necessitate a more robust, adaptive, and automated approach to security enforcement. In this paper, we present a framework leveraging large language models (LLMs) for automating attack mitigation policy compliance through an innovative combination of in-context learning and retrieval-augmented generation (RAG). We begin by describing how our system collects and manages both tool and API specifications, storing them in a vector database to enable efficient retrieval of relevant information. We then detail the architectural pipeline that first decomposes high-level mitigation policies into discrete tasks and subsequently translates each task into a set of actionable API calls. Our empirical evaluation, conducted using publicly available CTI policies in STIXv2 format and Windows API documentation, demonstrates significant improvements in precision, recall, and F1-score when employing RAG compared to a non-RAG baseline.

Problem

• Automating security policies using LLMs for cyber threats
• Leveraging RAG to improve policy compliance precision
• Translating mitigation policies into actionable API calls

Innovation

• Leveraging LLMs for automated security policy enforcement
• Combining in-context learning with retrieval-augmented generation
• Decomposing policies into actionable API calls via vector database