Correct but Slow: An Empirical Study of the GPU Kernel Evaluation Gap in Modern Domain-Specific Languages

2026-07-05

AI Summary

This work addresses the performance gap between verified kernels generated by modern GPU domain-specific languages (e.g., Triton, TileLang) and highly optimized library implementations, which often renders the former impractical for deployment despite correctness. Through a systematic evaluation of 22 kernels on A100 and GH200 GPUs, the study demonstrates that correctness verification alone fails to capture substantial performance deficiencies. It distinguishes between remediable code-level inefficiencies and inherent structural bottlenecks, and proposes two lightweight screening metrics based on efficiency ratios and Roofline model analysis. Experimental results reveal that some generated kernels are over 300× slower than PyTorch baselines, while the proposed methods accurately identify all inefficient yet correct kernels, thereby avoiding the overhead of exhaustive benchmarking.

Abstract

Modern GPU domain-specific languages (DSLs), such as Triton and TileLang, are increasingly used to implement specialized deep-learning kernels and as target languages for automated kernel-generation systems. Existing DSL-kernel evaluations establish correctness through reference-based numerical validation -- necessary, but silent on replacement quality: a functionally valid kernel may still fall far below the throughput of the optimized library operator it is intended to replace. We study this correctness-performance gap using 22 Triton and TileLang kernels from five operator categories on NVIDIA A100 and GH200 GPUs, asking whether correctness-based evaluation identifies kernels unsuitable as library replacements, why such failures occur, and how they can be detected without exhaustive benchmark coverage. The study yields three results. \emph{First}, correctness-based evaluation can admit severe slowdowns: an idiomatic TileLang LayerNorm kernel passes KernelBench's correctness check while running more than 300$\times$ slower than the PyTorch baseline. \emph{Second}, the causes differ by kernel family. TileLang normalization and reduction slowdowns are mainly repairable authoring defects, such as sequential reductions and unnecessary dtype conversions, whereas convolution and large general matrix multiplication (GEMM) retain residual gaps after optimization due to code-generation and autotuning-coverage limits; vendor-library algorithm selection contributes only marginally. \emph{Third}, two lightweight checks -- library-relative efficiency and roofline utilization -- are complementary screening criteria: together they flag every functionally valid but inefficient kernel in our suite and separate repairable authoring defects from structural residuals.

Problem

• GPU kernel evaluation
• correctness-performance gap
• domain-specific languages
• kernel efficiency
• library replacement

Innovation

• GPU kernel evaluation
• correctness-performance gap
• domain-specific languages
• roofline model
• kernel optimization