Building a Robust Risk-Based Access Control System to Combat Ransomware's Capability to Encrypt: A Machine Learning Approach

2026-01-23

AI Summary

This work proposes a risk-based access control architecture to counter unauthorized encryption by ransomware in Linux systems, enabling real-time blocking while preserving legitimate cryptographic operations. The approach uniquely integrates function-level kernel tracing—leveraging ftrace’s function_graph tracer—with interpretable policy rules, combining a supervised learning model and SELinux Boolean policies to achieve fine-grained, low-overhead, and explainable control over encryption activities. Evaluated under I/O-intensive workloads, the prototype demonstrates high detection accuracy and rapid response, effectively distinguishing malicious from benign encryption without incurring the performance penalties associated with virtualization or sandboxing mechanisms.

Technology Category

• Machine Learning: Kernel Methods
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Security and Privacy: Applications of cryptography
• Responsible Web: Measurement, analysis, and circumvention of Web censorship
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Ransomware core capability, unauthorized encryption, demands controls that identify and block malicious cryptographic activity without disrupting legitimate use. We present a probabilistic, risk-based access control architecture that couples machine learning inference with mandatory access control to regulate encryption on Linux in real time. The system builds a specialized dataset from the native ftrace framework using the function_graph tracer, yielding high-resolution kernel-function execution traces augmented with resource and I/O counters. These traces support both a supervised classifier and interpretable rules that drive an SELinux policy via lightweight booleans, enabling context-sensitive permit/deny decisions at the moment encryption begins. Compared to approaches centered on sandboxing, hypervisor introspection, or coarse system-call telemetry, the function-level tracing we adopt provides finer behavioral granularity than syscall-only telemetry while avoiding the virtualization/VMI overhead of sandbox-based approaches. Our current user-space prototype has a non-trivial footprint under burst I/O; we quantify it and recognize that a production kernel-space solution should aim to address this. We detail dataset construction, model training and rule extraction, and the run-time integration that gates file writes for suspect encryption while preserving benign cryptographic workflows. During evaluation, the two-layer composition retains model-level detection quality while delivering rule-like responsiveness; we also quantify operational footprint and outline engineering steps to reduce CPU and memory overhead for enterprise deployment. The result is a practical path from behavioral tracing and learning to enforceable, explainable, and risk-proportionate encryption control on production Linux systems.

Problem

• ransomware
• unauthorized encryption
• access control
• machine learning
• Linux security

Innovation

• risk-based access control
• function-level tracing
• machine learning for security
• SELinux policy
• ransomware mitigation