ACT: Automatically Generating Compiler Backends from Tensor Accelerator ISA Descriptions

2025-10-10

AI Summary

Tensor accelerators (NPUs) evolve rapidly, but manual compiler backend development is costly and time-consuming, severely hindering software ecosystem growth. Method: This paper proposes an automated compiler backend generation framework grounded in a formal, extensible, and parameterized ISA specification. It integrates parametric e-graph–based instruction selection with constraint-programming–driven memory allocation to ensure semantic correctness and completeness, while supporting user-programmable storage and complex parameterized instructions. Contribution/Results: The framework has successfully generated production-quality backends for three industrial and academic accelerators. Generated code matches or exceeds the performance of hand-optimized libraries, with manageable compilation overhead. It significantly improves software portability and development agility for emerging hardware architectures.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Tensor compilers play a key role in enabling high-performance implementations of deep learning workloads. These compilers rely on existing CPU and GPU code generation backends to generate device-specific code. Recently, many tensor accelerators (neural processing units) have been proposed to further accelerate these workloads. Compared to commodity hardware, however, most of the proposed tensor accelerators do not have compiler backends with code generation support. Moreover, the accelerator designs are subject to fast iteration cycles, making it difficult to manually develop compiler backends similar to commodity hardware platforms. Therefore, to increase adoption and enable faster software development cycles for novel tensor accelerator designs, we need to make the compiler backend construction process more agile. To address this gap, we introduce ACT, a compiler backend generator that automatically generates compiler backends for tensor accelerators, given just the instruction set architecture (ISA) descriptions. We first formally specify the compiler backend generation problem that introduces a novel specification for describing tensor accelerator ISAs. Next, we design ACT such that it supports user-programmable memories and complex parameterized instructions that are prevalent in tensor accelerators. ACT uses a novel parameterized equality saturation-based instruction selection phase and a constraint programming-based memory allocation phase. We prove that compiler backends generated by ACT are sound and complete. Finally, we generate compiler backends for three accelerator platforms from industry and academia, and show that they match or outperform code written using hand-optimized kernel libraries while maintaining low compilation overheads.

Problem

• Automating compiler backend generation for tensor accelerators
• Addressing fast iteration cycles in accelerator hardware design
• Enabling high-performance code without manual backend development

Innovation

• Automatically generates compiler backends from ISA descriptions
• Uses parameterized equality saturation for instruction selection
• Employs constraint programming for memory allocation