PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS

2025-12-19

AI Summary

To address the poor interpretability of graph neural network (GNN)-based intrusion detection systems (IDS) for temporal provenance graphs—hindering analyst trust in security operations centers (SOCs)—this paper proposes VA-TGExplainer, the first lightweight post-hoc explanation framework tailored to dynamic provenance graphs. Methodologically, it integrates GraphMask and GNNExplainer while introducing novel variational temporal modeling and causal subgraph extraction, enabling reproducible, low-overhead explanations under memory constraints. Evaluated on the CADETS E3 dataset, VA-TGExplainer generates per-event explanations in only 3–5 seconds, accurately identifying critical causal subgraphs and discriminative edges—such as malicious file operations and anomalous network flows—that trigger alerts. This significantly enhances the transparency and credibility of IDS decisions.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Security and Privacy: Data transparency and provenance

Abstract

Modern intrusion detection systems (IDS) leverage graph neural networks (GNNs) to detect malicious activity in system provenance data, but their decisions often remain a black box to analysts. This paper presents a comprehensive XAI framework designed to bridge the trust gap in Security Operations Centers (SOCs) by making graph-based detection transparent. We implement this framework on top of KAIROS, a state-of-the-art temporal graph-based IDS, though our design is applicable to any temporal graph-based detector with minimal adaptation. The complete codebase is available at https://github.com/devang1304/provex.git. We augment the detection pipeline with post-hoc explanations that highlight why an alert was triggered, identifying key causal subgraphs and events. We adapt three GNN explanation methods - GraphMask, GNNExplainer, and a variational temporal GNN explainer (VA-TGExplainer) - to the temporal provenance context. These tools output human-interpretable representations of anomalous behavior, including important edges and uncertainty estimates. Our contributions focus on the practical integration of these explainers, addressing challenges in memory management and reproducibility. We demonstrate our framework on the DARPA CADETS Engagement 3 dataset and show that it produces concise window-level explanations for detected attacks. Our evaluation reveals that the explainers preserve the TGNN's decisions with high fidelity, surfacing critical edges such as malicious file interactions and anomalous netflows. The average explanation overhead is 3-5 seconds per event. By providing insight into the model's reasoning, our framework aims to improve analyst trust and triage speed.

Problem

• Enhancing analyst trust in black-box GNN-based intrusion detection systems
• Providing transparent, interpretable explanations for provenance-based security alerts
• Integrating XAI methods to identify key causal subgraphs in temporal graphs

Innovation

• Explainable AI framework for graph-based intrusion detection
• Adapts three GNN explanation methods to temporal provenance data
• Produces human-interpretable subgraphs and uncertainty estimates