AttackLLM: LLM-based Attack Pattern Generation for an Industrial Control System

2025-04-05

AI Summary

The industrial control systems (ICS) security community lacks high-quality, diverse attack pattern datasets, hindering robust evaluation of anomaly detection models. Existing datasets rely heavily on scarce physical testbeds and costly domain expertise, severely limiting scalability. Method: This paper introduces the first multi-agent large language model (LLM)-driven attack pattern generation framework specifically designed for ICS security. It integrates protocol-specific knowledge injection, attack semantic modeling, and collaborative prompt engineering—eliminating dependence on real-world testbeds or manual prior knowledge. Contribution/Results: The generated attack samples surpass human-annotated ones in diversity, semantic plausibility, and protocol compliance. They significantly enhance the comprehensiveness and reliability of anomaly detection model evaluation. The framework demonstrates strong scalability and has been validated on mainstream protocols including Modbus/TCP, exhibiting clear potential for industrial deployment.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Agent Communication

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Malicious examples are crucial for evaluating the robustness of machine learning algorithms under attack, particularly in Industrial Control Systems (ICS). However, collecting normal and attack data in ICS environments is challenging due to the scarcity of testbeds and the high cost of human expertise. Existing datasets are often limited by the domain expertise of practitioners, making the process costly and inefficient. The lack of comprehensive attack pattern data poses a significant problem for developing robust anomaly detection methods. In this paper, we propose a novel approach that combines data-centric and design-centric methodologies to generate attack patterns using large language models (LLMs). Our results demonstrate that the attack patterns generated by LLMs not only surpass the quality and quantity of those created by human experts but also offer a scalable solution that does not rely on expensive testbeds or pre-existing attack examples. This multi-agent based approach presents a promising avenue for enhancing the security and resilience of ICS environments.

Problem

• Generating attack patterns for ICS robustness evaluation
• Overcoming scarcity of ICS testbeds and expert data
• Improving anomaly detection with scalable LLM-generated attacks

Innovation

• LLM-based attack pattern generation
• Multi-agent approach for ICS security
• Data-centric and design-centric methodologies