Network Design for Wafer-Scale Systems with Wafer-on-Wafer Hybrid Bonding

2026-03-05

AI Summary

This work addresses the communication bottleneck imposed by off-chip bandwidth limitations in large language models, where data movement incurs significant overhead. It presents the first systematic study of how chiplet physical placement on wafer-scale integration affects interconnect network performance. Leveraging wafer-to-wafer hybrid bonding and a 2D mesh topology, the authors propose four novel layout strategies—Aligned, Interleaved, Rotated, and Contoured. Experimental results demonstrate that these strategies can improve communication throughput by up to 250%, reduce latency by 36%, and lower energy consumption per byte transferred by as much as 38%, thereby substantially enhancing the communication efficiency of wafer-scale systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Multiagent Systems: Agent Communication

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Transformer-based large language models are increasingly constrained by data movement as communication bandwidth drops sharply beyond the chip boundary. Wafer-scale integration using wafer-on-wafer hybrid bonding alleviates this limitation by providing ultra-high bandwidth between reticles on bonded wafers. In this paper, we investigate how the physical placement of reticles on wafers influences the achievable network topology and the resulting communication performance. Starting from a 2D mesh-like baseline, we propose four reticle placements (Aligned, Interleaved, Rotated, and Contoured) that improve throughput by up to 250%, reduce latency by up to 36%, and decrease energy per transmitted byte by up to 38%.

Problem

• Wafer-scale integration
• Network topology
• Reticle placement
• Communication performance
• Data movement

Innovation

• wafer-scale integration
• wafer-on-wafer hybrid bonding
• reticle placement
• network topology
• communication performance