🤖 AI Summary
This work addresses the inefficiency of GPU kernel tuning, which traditionally relies on repeatedly recompiling full applications and manually isolating kernels—a process that is both tedious and error-prone. The authors propose an automated approach that intercepts HSA runtime dispatches to capture device memory snapshots, kernel source code, and launch configurations, thereby generating minimal, self-contained reproducer projects that can be independently compiled and validated. Their method is the first to support automatic extraction of both HIP and Triton kernels, preserving virtual address fidelity in memory snapshots and retaining JIT tuning configurations for Triton, enabling recompilation without the original build system. Evaluated across six real-world workloads, the technique successfully extracts snapshots ranging from 152 MB to 30 GB, achieving a 13.6× speedup in kernel iteration time for llama.cpp and significantly accelerating tuning and generative optimization workflows.
📝 Abstract
Iterative GPU kernel tuning is bottlenecked by the scale of the applications that host the kernels. Rapid iteration requires isolating the kernel so it can be edited, recompiled, and validated without rebuilding the full application -- but manual isolation requires reconstructing build flags, dispatch configuration, and runtime inputs by hand, so developers usually settle for slow in-place edits.
We present Kerncap, an automated kernel extraction tool that intercepts dispatches at the HSA runtime for both HIP and Triton, bridging Triton's JIT-only metadata into HSA-level capture via a lightweight Python compile-hook shim. Kerncap performs an address-space closure of all device memory -- a virtual-address-faithful snapshot that preserves embedded device pointers without DWARF metadata or pointer chasing -- locates kernel sources, and emits self-contained reproducer projects. HIP reproducers use a Clang VFS overlay for source-level recompilation without modifying the original build system; Triton reproducers are tuning-pinned, binding the captured autotuner configuration into the artifact to preserve the JIT kernel's numerical contract.
Across six real-world HIP and Triton workloads spanning traditional HPC and ML domains on three AMD GPU architectures (CDNA2, CDNA3, RDNA3), \textsc{Kerncap} extracts and validates kernels from snapshots ranging from 152~MB to 30~GB -- including a VA-faithful capture of vLLM's Mixture-of-Experts weight pool reached through pointer indirection. On our llama.cpp case study, Kerncap's edit-recompile-validate loop achieves a 13.6x speedup over the traditional workflow, reducing kernel isolation from a multi-hour process to a single command. The resulting reproducers also serve as a substrate for autotuning agents and LLM-driven kernel generators that need rapid, isolated evaluation of candidates.