PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX

2026-08-18

AI Summary

研究通过PTXBench评估和适应大型语言模型以使用特定架构的PTX进行GPU内核优化，但效果不一，进一步采用监督微调改进模型性能。

Abstract

We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target instructions execute at runtime, and speedup over frontier libraries across GEMM and attention workloads on H100 and B200 GPUs. Our evaluation shows that architecture-specific PTX capability remains uneven: success rates fall substantially on complex attention backward workloads, and executing the target instructions does not necessarily translate into competitive performance. No evaluated model consistently matches frontier libraries across the suite. We further adapt Qwen3.6-27B using supervised fine-tuning. Repair-conditioned training improves several tasks, but generalization remains uneven; data coverage, balance, and the quality of the reasoning teacher matter in addition to dataset size. PTXBench provides an auditable testbed for measuring and improving LLMs' ability to exploit evolving GPU architectures.

Problem

• LLMs
• PTX
• GPU Kernel Optimization
• Functional Correctness
• Speedup

Innovation

• PTXBench
• architecture-specific PTX
• supervised fine-tuning
• GPU kernel optimization