HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization

2026-08-21

AI Summary

为解决GPU内核优化问题，提出HIERA框架，通过跨实现空间规划和迭代优化提升效率与性能。

Abstract

High-performance GPU kernels underpin modern deep learning and scientific computing. As workloads become increasingly diverse and GPU hardware evolves rapidly, developing efficient methods for automated GPU kernel generation and optimization has become increasingly important. Existing LLM-based methods typically optimize within a fixed implementation space, limiting either optimization flexibility or search efficiency. We propose \textsc{HIERA}, a hierarchical search-space planning framework for GPU kernel optimization. \textsc{HIERA} constructs contract-augmented task specifications, selects an appropriate implementation space across PyTorch operators, CUDA libraries, and custom CUDA kernels, and uses profiling feedback and expert knowledge to guide structured iterative refinement. Experiments on KernelBench across multiple various workload levels and base LLMs show that \textsc{HIERA} delivers stronger overall implementation validity, sample efficiency, and optimization performance than existing training-free methods, while remaining competitive with the training-based CUDA-L1 without additional model training. A case study on a specialized stencil operator from scientific computing further achieves a \(1.53\times\) speedup over cuDNN, demonstrating the potentiality of the general framework beyond standard machine-learning workloads.

Problem

• GPU kernel optimization
• workload diversity
• implementation space
• search efficiency
• automated generation

Innovation

• hierarchical search-space planning
• workload-aware
• automated GPU kernel generation
• profiling feedback
• structured iterative refinement