FractalSync: Lightweight Scalable Global Synchronization of Massive Bulk Synchronous Parallel AI Accelerators

2025-06-13

AI Summary

To address the global barrier synchronization challenge across massive processing elements (PEs) — arranged in tile grids ranging from 2×2 to 16×16 — on ultra-large-scale heterogeneous many-core AI accelerators under the Bulk Synchronous Parallel (BSP) model, this work proposes a fractal hierarchical hardware synchronization mechanism. The mechanism fully offloads barrier coordination to dedicated hardware, eliminating software overhead and achieving near-constant latency (≤3.2 cycles) at 1 GHz, with silicon area overhead <0.01%. Implemented on a RISC-V + matrix-multiply coprocessor tile architecture, it integrates custom synchronization units and a DMA-enhanced mesh NoC, replacing conventional atomic memory operation (AMO)-based software barriers. Compared to state-of-the-art AMO schemes, it reduces synchronization latency by up to 43×. The design has been taped out and verified, achieving full-chip timing closure for the MAGIA accelerator.

Technology Category

• Machine Learning: Hardware-aware ML
• Multiagent Systems: Coordination and Collaboration
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Agentic search
• Security and Privacy: Large-scale security measurements

Abstract

The slow-down of technology scaling and the emergence of Artificial Intelligence (AI) workloads have led computer architects to increasingly exploit parallelization coupled with hardware acceleration to keep pushing the performance envelope. However, this solution comes with the challenge of synchronization of processing elements (PEs) in massive heterogeneous many-core platforms. To address this challenge, we propose FractalSync, a hardware accelerated synchronization mechanism for Bulk Synchronous Parallel (BSP) systems. We integrate FractalSync in MAGIA, a scalable tile-based AI accelerator, with each tile featuring a RISC-V-coupled matrix-multiplication (MatMul) accelerator, scratchpad memory (SPM), and a DMA connected to a global mesh Network-on-Chip (NoC). We study the scalability of the proposed barrier synchronization scheme on tile meshes ranging from 2x2 PEs to 16x16 PEs to evaluate its design boundaries. Compared to a synchronization scheme based on software atomic memory operations (AMOs), the proposed solution achieves up to 43x speedup on synchronization, introducing a negligible area overhead (<0.01%). FractalSync closes timing at MAGIA's target 1GHz frequency.

Problem

• Synchronizing massive heterogeneous many-core AI accelerators
• Scalable global synchronization for Bulk Synchronous Parallel systems
• Reducing synchronization overhead in large-scale parallel processing

Innovation

• Hardware-accelerated synchronization for BSP systems
• Scalable tile-based AI accelerator integration
• Negligible area overhead with high speedup