A System Level Compiler for Massively-Parallel, Spatial, Dataflow Architectures
To address the lack of efficient and portable compilers for spatial dataflow architectures—such as the Cerebras Wafer-Scale Engine—this paper introduces MACH, a system-level compiler. Methodologically, MACH introduces (i) the first virtual machine abstraction tailored to spatial architectures; (ii) a domain-specific language (DSL) with NumPy semantics, backed by a multi-level intermediate representation (IR); and (iii) an extensible multi-target lowering framework that enables co-compilation across unified memory systems and spatial hardware. By bridging high-level tensor computations—including dense NumPy operations—to low-level Wafer-Scale Engine instructions, MACH achieves end-to-end compilation while preserving semantic expressiveness. Evaluation demonstrates substantial improvements in programming productivity and cross-architecture portability, without sacrificing performance or correctness.