CyberCScope: Mining Skewed Tensor Streams and Online Anomaly Detection in Cybersecurity Systems

2025-03-02

AI Summary

To address the challenge of real-time multi-type intrusion detection and behavioral evolution tracking in high-order temporal tensor streams—characterized by heterogeneous attributes including categorical and skewed continuous features—this paper proposes the first hybrid skewed infinite-/finite-dimensional tensor stream decomposition framework. Our method explicitly decouples heterogeneous attributes: it models categorical attributes via discrete structural representations and handles skewed continuous attributes through a robust distributional adaptation mechanism; furthermore, it enables dynamic anomaly pattern discovery and interpretable behavioral summarization via online incremental decomposition. Extensive experiments on large-scale real-world network datasets demonstrate that our approach significantly outperforms existing state-of-the-art methods, achieving breakthrough improvements in detection accuracy, timeliness, and interpretability.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Cybersecurity systems are continuously producing a huge number of time-stamped events in the form of high-order tensors, such as {count; time, port, flow duration, packet size, . . . }, and so how can we detect anomalies/intrusions in real time? How can we identify multiple types of intrusions and capture their characteristic behaviors? The tensor data consists of categorical and continuous attributes and the data distributions of continuous attributes typically exhibit skew. These data properties require handling skewed infinite and finite dimensional spaces simultaneously. In this paper, we propose a novel streaming method, namely CyberCScope. The method effectively decomposes incoming tensors into major trends while explicitly distinguishing between categorical and skewed continuous attributes. To our knowledge, it is the first to compute hybrid skewed infinite and finite dimensional decomposition. Based on this decomposition, it streamingly finds distinct time-evolving patterns, enabling the detection of multiple types of anomalies. Extensive experiments on large-scale real datasets demonstrate that CyberCScope detects various intrusions with higher accuracy than state-of-the-art baselines while providing meaningful summaries for the intrusions that occur in practice.

Problem

• Detect real-time anomalies in high-order tensor streams.
• Identify multiple intrusion types and their behaviors.
• Handle skewed continuous and categorical data attributes.

Innovation

• Decomposes tensors into major trends
• Handles skewed infinite and finite dimensions
• Detects multiple types of anomalies streaming