The Art of Closed-Formula Defaults: Search-Free Code Generation for Tensor Operators

2026-09-09

AI Summary

本文通过提供一种从操作符规格到GPU指令的分层代码生成工具，解决了大型语言模型推理中GPU内核优化问题。

Abstract

Agentic search and automated optimization of GPU kernels are powerful tools for large language model inference. Their effectiveness, however, depends not on the sophistication of the search itself, but on the clarity of the optimization problem being solved. We provide an application-first approach that drives a hierarchical code generation tool from operator specifi cation down to GPU instructions, and show that a clearly defined computational model makes the optimization problem tractable.

Problem

• Agentic search
• Automated optimization
• GPU kernels
• Large language model inference
• Computational model

Innovation

• search-free code generation
• tensor operators
• hierarchical code generation
• computational model