Lighthouse: An Open Research Framework for Optical Data Center Networks

2024-11-27

arXiv.org

AI Summary

Existing optical datacenter networks (DCNs) suffer from tight hardware-software coupling, hindering independent evolution of control software and optical hardware. Method: This paper introduces Lighthouse, an open and extensible research framework that pioneers the “time-flow table” abstraction as a universal hardware-software interface to decouple control logic from optical hardware. It implements a P4-programmable control plane, designs dynamic optical circuit scheduling algorithms, and develops a multi-architecture adaptation middleware. Contribution/Results: Lighthouse achieves, for the first time on commercial optical devices, sub-2-μs optical circuit setup latency—the lowest reported to date. It validates six optical architecture prototypes on a dedicated optical testbed and demonstrates high throughput and low latency in benchmarking across a 108-switch Top-of-Rack (ToR) topology. Already deployed to support multiple emerging research initiatives, Lighthouse provides foundational infrastructure for standardization and ecosystem development of optical DCNs.

Technology Category

• Search and Optimization: Distributed Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Data transparency and provenance

Abstract

Optical data center networks (DCNs) are emerging as a promising design for cloud infrastructure. However, existing optical DCN architectures operate as closed ecosystems, tying software solutions to specific optical hardware. We introduce Lighthouse, an open research framework that decouples software from hardware, allowing them to evolve independently. Central to Lighthouse is the time-flow table abstraction, serving as a common interface between optical hardware and software. We develop Lighthouse on programmable switches, achieving a minimum optical circuit duration of 2 {mu}s, the shortest duration realized by commodity devices to date. We demonstrate Lighthouse's generality by implementing six optical architectures on an optical testbed and conducted extensive benchmarks on a 108-ToR setup, highlighting system efficiency. Additionally, we present case studies that identify potential research topics enabled by Lighthouse.

Problem

• Decouples software from hardware in optical DCNs
• Provides common interface for optical hardware and software
• Enables diverse applications with efficient queue management

Innovation

• Decouples software from optical hardware
• Uses time-flow table abstraction interface
• Re-architects queue management backend system