Harvest: Adaptive Photonic Switching Schedules for Collective Communication in Scale-up Domains

2026-02-09

AI Summary

This work addresses the challenge of optical circuit reconfiguration in silicon photonic interconnects for inter-chip collective communication, where trade-offs among reconfiguration overhead, latency, and congestion must be carefully balanced. The authors propose Harvest, a novel approach that, for the first time, formulates photonic reconfiguration scheduling as an optimization problem, automatically generating optimal schedules for arbitrary collective communication algorithms to minimize total communication completion time. Notably, for the Recursive Doubling algorithm, Harvest derives a closed-form optimal schedule without requiring a solver and adapts dynamically to photonic device reconfiguration delays. Integrating dynamic programming, topology-aware optimization, and multi-granularity simulation, Harvest significantly outperforms both static interconnects and incremental reconfiguration baselines across diverse collective operations, with efficacy validated through simulation and experiments on a commercial GPU platform.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Algorithm Configuration
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

As chip-to-chip silicon photonics gain traction for their bandwidth and energy efficiency, their circuit-switched nature raises a fundamental question for collective communication: when and how should the interconnect be reconfigured to realize these benefits? Establishing direct optical paths can reduce congestion and propagation delay, but each reconfiguration incurs non-negligible overhead, making naive per-step reconfiguration impractical. We present Harvest, a systematic approach for synthesizing topology reconfiguration schedules that minimize collective completion time in photonic interconnects. Given a collective communication algorithm and its fixed communication schedule, Harvest determines how the interconnect should evolve over the course of the collective, explicitly balancing reconfiguration delay against congestion and propagation delay. We reduce the synthesis problem into a dynamic program with an underlying topology optimization subproblem and show that the approach applies to arbitrary collective communication algorithms. Furthermore, we exploit the algorithmic structure of a well-known AllReduce algorithm (Recursive Doubling) to synthesize optimal reconfiguration schedules without using any optimizers. By parameterizing the formulation using reconfiguration delay, Harvest naturally adapts to various photonic technologies. Using packet-level and flow-level evaluations, as well as hardware emulation on commercial GPUs, we show that the schedules synthesized by Harvest significantly reduce collective completion time across multiple collective algorithms compared to static interconnects and reconfigure-every-step baselines.

Problem

• photonic interconnects
• collective communication
• topology reconfiguration
• reconfiguration overhead
• completion time

Innovation

• photonic interconnects
• collective communication
• topology reconfiguration
• dynamic programming
• AllReduce