Beyond Collection: Measuring the Detection Efficacy of Modern Security Logging Standards

2026-05-06

AI Summary

This study addresses the lack of systematic evaluation of mainstream security logging standards in terms of their effectiveness for threat detection. The authors propose a scalable and reproducible assessment methodology based on an automated Security Exploit Telemetry Collection (SETC) framework, which reproduces 50 remote code execution vulnerabilities in containerized environments. Using this approach, they comparatively evaluate the telemetry completeness and attack detectability of widely adopted standards—including Common Information Model (CIM), Open Cybersecurity Schema Framework (OCSF), and Elastic Common Schema (ECS). The experiments quantitatively measure each standard’s detection efficacy, revealing significant disparities in coverage of critical attack indicators and identifying notable gaps. These findings provide empirical guidance for security practitioners in selecting appropriate logging standards to enhance threat detection capabilities.

Abstract

Effective security logging is crucial for the timely and accurate detection of cyber threats; however, the relative effectiveness of various industry-standard logging frameworks remains understudied. This paper addresses this critical gap by presenting the first systematic evaluation of modern security logging standards utilizing a novel methodology built upon the automated Security Exploit Telemetry Collection (SETC) framework. SETC systematically generates reproducible exploit scenarios in containerized environments, collecting rich telemetry across multiple logging standards, including CIM (Common Information Model), OCSF (Open Cybersecurity Schema Framework), and ECS (Elastic Common Schema). The detection efficacy of each logging standard is quantified by measuring telemetry completeness and exploit detectability across standardized logs through detailed experiments involving 50 diverse remote code execution vulnerabilities. The resulting findings identify critical gaps and reveal significant differences in logging standards' abilities to capture key attack indicators. Our contributions include a novel evaluation methodology that enables scalable and reproducible analysis of exploit telemetry, as well as new findings that provide clear, evidence-based guidance for security practitioners to make informed decisions about adopting logging standards.

Problem

• security logging
• detection efficacy
• logging standards
• cyber threat detection
• telemetry completeness

Innovation

• Security Logging Standards
• Detection Efficacy
• Automated Telemetry Collection
• Exploit Detectability
• Containerized Evaluation