SoK: Reshaping Research on Network Intrusion Detection Systems

2026-04-19

AI Summary

This study addresses the significant gap between academic research and real-world deployment in network intrusion detection systems (NIDS), which stems from a lack of consensus on the fundamental characteristics of NIDS and consequently leads to inconsistent evaluation benchmarks. Through a systematic survey (SoK), this work formally defines the intrinsic properties of NIDS, critically examines prevailing evaluation methodologies, and employs reproducible case studies to expose the disconnect between theoretical research and operational practice. Building on these insights, the paper proposes foundational principles and concrete recommendations for reframing NIDS research through the lens of security operations, aiming to align academic inquiry more closely with real-world requirements and provide actionable methodological guidance for future work.

Technology Category

• Search and Optimization: Evaluation and Analysis
• Multiagent Systems: Other Foundations of Multi Agent Systems
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Network Intrusion Detection Systems (NIDS) have been studied for decades. Hundreds of papers have, e.g., proposed ways to enhance, harden or bypass NIDS. However, the findings of prior literature are hardly reflected in real-world operational contexts. Such a disconnection is problematic for research itself: it is unclear what scenario envisioned by prior work can be used as a baseline for future advancements. We argue that a key reason for this disconnection is a fundamental misunderstanding of intrinsic characteristics of NIDS. For instance, the fact that a compromised NIDS cannot be expected to work well; the fact that some evaluations are done without carrying out any experiment in a (even synthetic) "real" network; the fact that security operators triage high-level reports -- and not individual samples flagged by some classifier. In this SoK, which is primarily a reflective piece, we first constructively highlight such quintessential properties (without criticizing _any_ work by different authors) by stating three Assertions. Then, we provide recommendations -- further emphasized through an original and reproducible case study that challenges some established practices. Ultimately, we seek to lay a foundation to reshape research on NIDS.

Problem

• Network Intrusion Detection Systems
• research-practice gap
• operational context
• evaluation methodology
• baseline scenario

Innovation

• Network Intrusion Detection Systems
• Systematization of Knowledge
• Real-world Evaluation
• Security Operations
• Reproducible Case Study