Virtuoso: High Resource Utilization and {mu}s-scale Performance Isolation in a Shared Virtual Machine TCP Network Stack

2023-09-25

AI Summary

In virtualized environments, hierarchical network architectures incur high overhead and insufficient performance isolation, failing to meet stringent requirements for bandwidth, scalability, and microsecond-scale latency. This paper proposes a shared virtual network stack for VMs and containers, introducing the novel “shared yet strongly isolated” architecture. It features single-instance elastic CPU core binding, per-packet resource accounting and scheduling, a one-hop full-stack fast path (from TCP to vSwitch), kernel bypass, and zero-copy optimizations—establishing a single-layer, redundancy-free data plane. Strict multi-tenant performance isolation is guaranteed while maximizing CPU utilization and enabling deterministic latency control. Experimental evaluation demonstrates that, compared to state-of-the-art solutions, our design improves CPU efficiency by up to 82%, reduces end-to-end latency by up to 58%, and incurs only a 6.7% processing overhead relative to bare-metal performance.

Abstract

Virtualization improves resource efficiency and ensures security and performance isolation for cloud applications. Today, operators use a layered architecture with separate network stack instances in each VM and container connected to a virtual switch. Decoupling through layering reduces complexity, but induces performance and resource overheads at odds with increasing demands for network bandwidth, connection scalability, and low latency. We present Virtuoso, a new software network stack for VMs and containers. Virtuoso re-organizes the network stack to maximize CPU utilization, enforce isolation, and minimize processing overheads. We maximize utilization by running one elastically shared network stack instance on dedicated cores; we enforce isolation by performing central and fine-grained per-packet resource accounting and scheduling; we reduce overheads by building a single-layer data path with a one-shot fast-path incorporating all processing from the TCP transport layer through network virtualization and virtual switching. Virtuoso improves resource efficiency by up to 82%, latencies by up to 58% compared to other virtualized network stacks without sacrificing isolation, and keeps processing overhead within 6.7% of unvirtualized stacks.

Problem

• Improving CPU utilization in virtualized network stacks
• Ensuring performance isolation for cloud applications
• Reducing processing overheads in shared network stacks

Innovation

• Elastically shared network stack on dedicated cores
• Fine-grained per-packet resource accounting and scheduling
• Single-layer data path with one-shot fast-path