Accelerating Precise End-to-End Simulation: Latency-Sensitive Many-core System Modeling

2026-05-08

AI Summary

This work addresses the challenges posed by large language models to many-core architectures, particularly the demand for high parallelism and on-chip memory capacity, which render traditional RTL simulation inefficient for accurately modeling latency-sensitive interconnect behavior. The authors propose an end-to-end rapid modeling methodology that combines hierarchical interconnect modeling, timing-abstracted scratchpad memory (SPM) access, and NoC router remapping to preserve critical timing details while significantly simplifying non-essential hardware components. Evaluated on the TeraNoC platform—a 1024-core system with 4 MiB shared L1 SPM—the approach achieves up to 115× simulation speedup with less than 7% error, enables fine-grained performance profiling, and successfully guides optimizations in FlashAttention-2 to mitigate interconnect stalls. Furthermore, design space exploration of the NoC yields substantial throughput improvements.

Abstract

Modern large language model workloads put increasing demands on parallel compute capability and on-chip memory capacity, while also stressing fine-grained data movement and synchronization. These trends motivate exploring and designing many-core accelerators with tightly coupled scratchpad memory (SPM) for scalable compute and predictable, explicitly managed data access. However, this architectural shift raises two challenges: cycle-accurate register-transfer level (RTL) simulation becomes prohibitively slow as system complexity grows, and performance estimation requires precise modeling of latency-sensitive interconnect behavior. This paper presents a fast yet accurate end-to-end modeling approach for latency-sensitive many-core architectures, targeting large-scale instances such as TeraNoC with 1024 cores and a 4MiB globally shared L1 SPM. The approach captures timing behavior of latency-sensitive SPM accesses across multiple interconnect scales, while abstracting non-essential hardware details. Across diverse benchmarks, the model tracks a cycle-accurate RTL golden model with errors below 7%, while delivering up to 115x faster simulation. The framework also provides detailed profiling across processing elements and interconnect, enabling efficient end-to-end software development and hardware design exploration. Two case studies demonstrate its practicality: profiling-guided optimization of FlashAttention-2 to reduce interconnect stalls and synchronization overhead, and design space exploration of network-on-chip (NoC) router remapping to alleviate traffic imbalance and improve throughput.

Problem

• many-core
• latency-sensitive
• cycle-accurate simulation
• interconnect modeling
• scratchpad memory

Innovation

• many-core simulation
• scratchpad memory
• latency-sensitive modeling
• end-to-end performance estimation
• network-on-chip