Modeling Behavioral Preferences of Cyber Adversaries Using Inverse Reinforcement Learning

2025-05-02

AI Summary

Existing threat attribution methods rely on coarse-grained, static attacker profiles based on TTPs (Tactics, Techniques, and Procedures), failing to capture dynamic, fine-grained behavioral intent across attack stages and tools. Method: This paper proposes a novel system-level audit log modeling approach that infers attackers’ intrinsic behavioral preferences via inverse reinforcement learning (IRL) from low-level forensic logs. It constructs transferable behavioral fingerprints by jointly modeling attack provenance graphs and extracting state-action trajectories. Contribution/Results: Evaluated on real-world attack log datasets, the method identifies discriminative and robust preference patterns—significantly improving attribution accuracy. By capturing stable decision-making tendencies underlying heterogeneous tool usage and multi-stage operations, it overcomes the limitations of static TTP-based profiling and enables deeper, more generalizable behavioral characterization for threat attribution.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper presents a holistic approach to attacker preference modeling from system-level audit logs using inverse reinforcement learning (IRL). Adversary modeling is an important capability in cybersecurity that lets defenders characterize behaviors of potential attackers, which enables attribution to known cyber adversary groups. Existing approaches rely on documenting an ever-evolving set of attacker tools and techniques to track known threat actors. Although attacks evolve constantly, attacker behavioral preferences are intrinsic and less volatile. Our approach learns the behavioral preferences of cyber adversaries from forensics data on their tools and techniques. We model the attacker as an expert decision-making agent with unknown behavioral preferences situated in a computer host. We leverage attack provenance graphs of audit logs to derive a state-action trajectory of the attack. We test our approach on open datasets of audit logs containing real attack data. Our results demonstrate for the first time that low-level forensics data can automatically reveal an adversary's subjective preferences, which serves as an additional dimension to modeling and documenting cyber adversaries. Attackers' preferences tend to be invariant despite their different tools and indicate predispositions that are inherent to the attacker. As such, these inferred preferences can potentially serve as unique behavioral signatures of attackers and improve threat attribution.

Problem

• Modeling cyber adversaries' behavioral preferences using IRL
• Learning attacker preferences from forensics data and tools
• Identifying invariant preferences as unique behavioral signatures

Innovation

• Uses inverse reinforcement learning for adversary modeling
• Learns attacker preferences from forensics data
• Derives state-action trajectories from audit logs