Comprehensive Kernel Safety in the Spectre Era: Mitigations and Performance Evaluation (Extended Version)

2024-11-27

arXiv.org

AI Summary

This work addresses the breakdown of kernel ASLR under speculative execution and side-channel attacks in the Spectre era, revealing its inconsistency with Abadi et al.’s shared-memory assumption and highlighting prevalent isolated memory models and system-call communication channels in practice. We propose three verifiable secure execution mechanisms, formally defining— for the first time—the security boundary of ASLR within kernel memory isolation models. Furthermore, we design the first Spectre-era system-call speculation-resilience framework, enabling verifiable kernel-level side-channel immunity. Implemented and evaluated on Linux, our solution integrates retpoline, IBRS, and user-space access blocking; it incurs an average performance overhead of <3.2%, maintains bounded latency increases for critical system calls, and empirically demonstrates a feasible security–performance trade-off.

Technology Category

• Machine Learning: Hardware-aware ML
• Application Domains: Security
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

The efficacy of address space layout randomization has been formally demonstrated in a shared-memory model by Abadi et al., contingent on specific assumptions about victim programs. However, modern operating systems, implementing layout randomization in the kernel, diverge from these assumptions and operate on a separate memory model with communication through system calls. In this work, we relax Abadi et al.'s language assumptions while demonstrating that layout randomization offers a comparable safety guarantee in a system with memory separation. However, in practice, speculative execution and side-channels are recognized threats to layout randomization. We show that kernel safety cannot be restored for attackers capable of using side-channels and speculative execution, and introduce enforcement mechanisms that can guarantee speculative kernel safety for safe system calls in the Spectre era. We implement three suitable mechanisms and we evaluate their performance overhead on the Linux kernel.

Problem

• Evaluates kernel safety under Spectre attacks with side-channels
• Proposes mechanisms for speculative kernel safety in modern OS
• Assesses performance overhead of mitigations in Linux kernel

Innovation

• Relaxed language assumptions for kernel safety
• Introduced speculative kernel safety mechanisms
• Evaluated performance overhead on Linux kernel