On Kernel's Safety in the Spectre Era (Extended Version)

2024-06-11

arXiv.org

AI Summary

Spectre-class speculative execution and side-channel attacks undermine traditional ASLR, rendering kernel security difficult to formalize and verify. Method: This paper introduces, for the first time, a formal definition of kernel security that does not rely on address-space layout randomization (ASLR). We construct a shared/isolated memory model and precisely characterize speculative execution and side-channel threats, then rigorously prove that our security condition is both reasonably weak and sufficient to enforce secure system calls. Contribution/Results: Building on this foundation, we design a deployable defense mechanism that preserves practicality while providing the first formally verified alternative security foundation for kernels in the Spectre era—thereby enabling principled, mathematically grounded kernel hardening against microarchitectural attacks.

Technology Category

• Application Domains: Security
• Machine Learning: Kernel Methods
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures

Abstract

The efficacy of address space layout randomization has been formally demonstrated in a shared-memory model by Abadi et al., contingent on specific assumptions about victim programs. However, modern operating systems, implementing layout randomization in the kernel, diverge from these assumptions and operate on a separate memory model with communication through system calls. In this work, we relax Abadi et al.'s language assumptions while demonstrating that layout randomization offers a comparable safety guarantee in a system with memory separation. However, in practice, speculative execution and side-channels are recognized threats to layout randomization. We show that kernel safety cannot be restored for attackers capable of using side-channels and speculative execution and introduce a new condition, that allows us to formally prove kernel safety in the Spectre era. Our research demonstrates that under this condition, the system remains safe without relying on layout randomization. We also demonstrate that our condition can be sensibly weakened, leading to enforcement mechanisms that can guarantee kernel safety for safe system calls in the Spectre era.

Problem

• Analyzing kernel safety under Spectre attacks with side-channels
• Relaxing assumptions for layout randomization in separated memory
• Introducing new condition to formally prove Spectre-era kernel safety

Innovation

• Relaxing language assumptions for safety guarantees
• Introducing new condition for Spectre-era kernel safety
• Weakening condition for safe system call enforcement