Tide: A Split OS Architecture for Control Plane Offloading

2024-08-30

AI Summary

Facing efficiency bottlenecks in cloud servers amid the end of Moore’s Law, this paper proposes a Split OS architecture that offloads OS control-plane tasks to embedded ARM cores on SmartNICs. Our approach employs mechanism-policy separation, a low-overhead cross-device communication API, lightweight state synchronization, and PCIe latency avoidance—enabling, for the first time, microsecond-scale OS control-plane execution directly on SmartNICs, with performance matching or exceeding that of the host CPU. Evaluation shows: memory management offloading frees up 16 CPU cores; Stubby RPC reduces overhead by eight cores; and GCE virtual machine management throughput improves by 11.2%. This work demonstrates the feasibility and substantial benefits of hardware-accelerated control-plane offloading, establishing a new paradigm for enhancing cloud infrastructure efficiency.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search
• Planning, Routing, and Scheduling: Plan Execution and Monitoring

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Cloud, edge and content delivery systems for the Web
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

The end of Moore's Law is driving cloud providers to offload virtualization and the network data plane to SmartNICs to improve compute efficiency. Even though individual OS control plane tasks consume up to 5% of cycles across the fleet, they remain on the host CPU because they are tightly intertwined with OS mechanisms. Moreover, offloading puts the slow PCIe interconnect in the critical path of OS decisions. We propose Tide, a new split OS architecture that separates OS control plane policies from mechanisms and offloads the control plane policies onto a SmartNIC. Tide has a new host-SmartNIC communication API, state synchronization mechanism, and communication mechanisms that overcome the PCIe bottleneck, even for $mu$s-scale workloads. Tide frees up host compute for applications and unlocks new optimization opportunities, including machine learning-driven policies, scheduling on the network I/O path, and reducing on-host interference. We demonstrate that Tide enables OS control planes that are competitive with on-host performance for the most difficult $mu$s-scale workloads. Tide outperforms on-host control planes for memory management (saving 16 host cores), Stubby network RPCs (saving 8 cores), and GCE virtual machine management (11.2% performance improvement).

Problem

• Offload userspace system software to SmartNIC cores
• Optimize datacenter efficiency via host-SmartNIC resource management
• Improve scalability by rethinking system software placement

Innovation

• Offload userspace system software to SmartNIC
• Introduce new host-SmartNIC communication API
• Optimize resource management with SmartNIC ARM cores