Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation

2026-07-01

AI Summary

This work addresses the challenge of generating high-performance kernels for NPUs, which is hindered by implicit hardware constraints and stringent memory hierarchies that large language models—lacking hardware-specific priors—struggle to satisfy. To overcome this, we propose Hawk, a novel framework featuring a synergistic three-module architecture: it constructs a Triple-Part executable knowledge representation through runtime knowledge synthesis, employs bottleneck-aware two-dimensional retrieval (2D-Retrieval), and leverages execution feedback from real NPU runs to drive effect-guided semantic arbitration and knowledge distillation—all without requiring model retraining. Evaluated on real-world NPU workloads, Hawk improves code generation accuracy from 49.4% to 80.0% and achieves up to 2.2× speedup over state-of-the-art baselines.

Abstract

Developing high-performance kernels for Neural Processing Units (NPUs) is a critical industry bottleneck, requiring developers to manually navigate implicit hardware constraints and strict memory hierarchies. While large language models offer immense automation potential, they fail catastrophically on NPUs due to a fundamental lack of hardware-specific priors. Naively transplanting code snippets from similar NPU kernels may pass the compiler, but it consistently triggers runtime crashes and performance degradation by blindly violating underlying hardware constraints. To overcome this, we introduce Hawk, a training-free framework that harnesses hardware-aware knowledge through three core modules: (1) Run-Time Knowledge Synthesis Module, which employs a Triple-Part Executable Knowledge Representation to inherently couple the error context with executable semantics; (2) Bottleneck-Aware Knowledge Retrieval Module, which implements a 2D-Retrieval paradigm to project queries into orthogonal syntactic and hardware-aligned semantic spaces; and (3) Effect-Driven Knowledge Distillation Module, which leverages LLM-driven semantic arbitration to continuously distill the knowledge by pruning errors and consolidating redundancies based on the empirical execution feedback. Extensive evaluations on real-world NPU workloads demonstrate that Hawk elevates generation accuracy from 49.4% to 80.0%, while achieving up to a 2.2x execution speedup over state-of-the-art baselines.

Problem

• Neural Processing Units
• hardware constraints
• kernel generation
• performance degradation
• memory hierarchies

Innovation

• Hardware-Aware Knowledge
• NPU Kernel Generation
• Knowledge Distillation
• 2D-Retrieval
• Executable Knowledge Representation