Exploiting exttt{ftrace}'s exttt{function_graph} Tracer Features for Machine Learning: A Case Study on Encryption Detection

2025-12-04

AI Summary

This paper addresses the challenge of effectively detecting large-scale file encryption activities. We propose a system-level behavioral modeling and machine learning integration framework leveraging Linux kernel ftrace’s function_graph tracer. By capturing fine-grained function call traces, we construct system call graphs and—novelty—the first systematic incorporation of native ftrace call graph structural features into machine learning tasks. We design a joint feature extraction method capturing both sequential call patterns and graph-topological properties, and employ a graph neural network coupled with a multi-class classifier for end-to-end modeling. Evaluated on real-world encryption scenarios, our approach achieves 99.28% classification accuracy and supports multi-label program identification. It significantly enhances automation and discriminative precision in system behavior anomaly detection, establishing a new paradigm for synergistic system tracing and AI-driven analysis.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper proposes using the Linux kernel ftrace framework, particularly the function graph tracer, to generate informative system level data for machine learning (ML) applications. Experiments on a real world encryption detection task demonstrate the efficacy of the proposed features across several learning algorithms. The learner faces the problem of detecting encryption activities across a large dataset of files, using function call traces and graph based features. Empirical results highlight an outstanding accuracy of 99.28 on the task at hand, underscoring the efficacy of features derived from the function graph tracer. The results were further validated in an additional experiment targeting a multilabel classification problem, in which running programs were identified from trace data. This work provides comprehensive methodologies for preprocessing raw trace data and extracting graph based features, offering significant advancements in applying ML to system behavior analysis, program identification, and anomaly detection. By bridging the gap between system tracing and ML, this paper paves the way for innovative solutions in performance monitoring and security analytics.

Problem

• Detecting encryption activities using system call traces
• Extracting graph-based features from Linux kernel ftrace data
• Applying machine learning for system behavior analysis and security

Innovation

• Using ftrace function graph tracer for system data generation
• Extracting graph-based features from function call traces
• Applying machine learning to system behavior and security analysis