PACT: Reducing Alert Fatigue in Low-Prevalence SOC Streams with Triggered Active Learning

2026-05-21

AI Summary

This study addresses alert fatigue in Security Operations Centers caused by excessive false positives within low-prevalence alert streams. To mitigate this issue, the authors propose PACT, a novel approach that integrates a Pareto-aware controller with trigger-based active learning. Building upon a frozen XGBoost-Focal filter, PACT introduces an adaptive window scoring offset mechanism and a hybrid sample acquisition strategy that innovatively combines threshold-relative uncertainty with high-confidence sampling. This design effectively balances false positive reduction against recall preservation, avoiding the performance collapse often induced by aggressive false positive minimization alone. Experimental results demonstrate that PACT reduces normalized false positive burden by 43% on the AIT-ADS dataset and by 21% on BOTSv1, while requiring 3.8× and 5.2× fewer analyst queries, respectively, compared to periodic random retraining.

Abstract

Security operations centers face persistent alert fatigue: in low-prevalence streams, even low false-positive rates generate substantial investigation load, while aggregate F1 scores obscure analyst burden. We introduce PACT, a Pareto-aware controller for triggered active learning, which wraps an already-deployed frozen XGBoost-Focal screener with an adaptive windowing score-shift trigger and a hybrid acquisition rule combining threshold-relative uncertainty with high-score sampling. On two public low-prevalence benchmarks, AIT-ADS (AIT Alert Data Set), and BOTSv1 (Boss of the SOC version 1), PACT attains the lowest benign-normalized false-positive (FP) burden among the adaptive methods tested. It reduces burden by 43% and 21%, respectively, relative to a frozen baseline, while using 3.8x and 5.2x fewer analyst queries than periodic uniform-random updating. A matched-trigger ablation controls trigger timing and shows that acquisition contributes beyond timing alone, at the cost of approximately ten percentage points of positive-window recall under free-running triggers. A frozen threshold-only baseline pushes FP lower still but collapses BOTSv1 recall by 55 percentage points. Under the evaluated workload assumptions, pure FP minimization trades unacceptable recall for that lower burden.

Problem

• alert fatigue
• low-prevalence
• security operations center
• false-positive burden
• analyst workload

Innovation

• triggered active learning
• alert fatigue
• adaptive windowing
• hybrid acquisition rule
• low-prevalence streams