Can SOC Operators Explain their Decisions while Triaging Alarms? A Real-World Study

2026-04-23

AI Summary

This study addresses a critical gap in explainability within Security Operations Center (SOC) workflows: while SOC analysts achieve high decision accuracy (83%) during alert triage, their post-hoc explanations align with the true root causes in only 39% of cases. Through a systematic literature review encompassing 257 papers and an empirical user study involving 12 SOC analysts, this work provides the first evidence of a significant disconnect between decision correctness and explanation fidelity. The findings underscore the urgent need for computational mechanisms that support accurate, causally grounded justifications for analyst decisions. By revealing this explanatory deficit, the research offers foundational insights for enhancing human–machine collaboration and bolstering trustworthiness in SOC decision-making processes.

Technology Category

• Reasoning under Uncertainty: Causality
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Security Operations Centers (SOCs) are pivotal in modern enterprises. Tasked to monitor complex network environments constantly under attack, SOCs can be active 24/7 and can include hundreds of operators supported by state-of-the-art technologies. Abundant research has studied the internal processes of SOCs, highlighting their pros and cons, as well as the challenges faced by SOC analysts -- such as dealing with the overwhelming number of false alarms triggered by automated security mechanisms. In this context, we wonder: given that "someone" must triage the alarms, and that such triaging must be grounded on established knowledge or evidence-based reasoning, can SOC employees justify why a certain decision was taken while triaging alarms? Answering such a research question (RQ) can better guide future efforts. We hence tackle this RQs. First, via a systematic literature review across 257 research documents, we provide evidence that such RQ received limited attention so far. Then, we partner-up with a real-world SOC and carry out a field study (n=12) with SOC employees. We show them real alarms raised in their SOC, and inquire whether such alarms are indicative of true security problems or not. Then, we ask to explain their decision. We found that while most analysts were able to separate "true from false" alarms (the decision was correct in 83% of the cases), a correct justification was hardly provided (only 39% of the provided explanations reflected the actual root cause). Ultimately, our results highlight the need for decision-support systems that help SOC analysts not only make the right call -- but also understand and articulate why it is right.

Problem

• SOC
• alarm triaging
• decision justification
• security operations
• explanation

Innovation

• explainable security
• alarm triaging
• SOC decision-making
• justification gap
• human factors in cybersecurity