Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents

2026-06-16

AI Summary

This work addresses the challenge of accurately predicting adversary (red team) strategies in partially observable network environments, where defenders face significant uncertainty. The authors propose a novel approach that integrates imitation learning, reinforcement learning, and neuro-symbolic systems—specifically leveraging learnable behavior trees. For the first time, imitation learning is employed to model red-team strategies within discrete state and action spaces, enabling high-fidelity inference of adversarial behavior patterns from limited observational data and defender actions alone. Experimental results demonstrate that the method effectively predicts a diverse range of attack strategies across multiple simulated scenarios, substantially enhancing the adaptability and decision-making accuracy of neuro-symbolic defensive agents.

Abstract

With sophisticated cyber-attacks becoming increasingly prevalent, modern networks require intelligent autonomous cyber-defense agents trained via Reinforcement Learning (RL). These agents employ neurosymbolic approaches such as behavior trees with learning-enabled components (LECs) to learn, reason, adapt, and implement security rules while maintaining critical operations. However, these autonomous networks are partially observable systems, i.e., the cyber-attacker's (red agent's) actions are not observable, making it difficult for the defender to predict red actions, learn red policies, or assess the attacker's intrusion levels. To address this, we propose a Policy Learning Technique using imitation learning to learn policies for partially observable RL agents with discrete states and discrete actions. We apply this technique in an autonomous cyber environment to predict red agent's actions from network observations and defender actions. Integrated with a neurosymbolic cyber-defense agent, our method effectively handles different red policies and achieves high prediction accuracy across diverse simulated scenarios.

Problem

• partially observable systems
• red agent policy
• cyber-defense
• action prediction
• autonomous cyber agents

Innovation

• imitation learning
• partially observable reinforcement learning
• neurosymbolic agents
• red team policy inference
• autonomous cyber defense