daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization

2026-06-15

AI Summary

This work addresses the challenge of efficiently reusing effective optimization skills in GPU kernel tuning by proposing a reinforcement learning–based co-evolutionary framework. For the first time, it jointly trains three agents—skill selection, policy generation, and skill summarization—within a shared large language model (LLM) backbone, ensuring skill validity through an execution-based verification mechanism. The framework integrates BM25 retrieval, LLM-based reranking, multi-round REINFORCE optimization, and structured supervised fine-tuning to enable end-to-end automatic generation and optimization of CUDA/Triton kernels. Evaluated on KernelBench, daVinci-kernel-14B achieves pass rates of 37.2%, 70.6%, and 32.2% at Level 1/2/3 under the Fast₁ threshold, significantly outperforming the previous state-of-the-art RL model, Dr.Kernel-14B.

Abstract

GPU kernel optimization represents a paradigm where functional correctness is assumed and execution efficiency is the objective. We present daVinci-kernel, a reinforcement learning framework that couples skill discovery with skill exploitation through a dynamically evolving skill library. daVinci-kernel jointly trains three agents sharing one LLM backbone: a Skill Selection Agent that retrieves relevant techniques via BM25 and LLM reranking, a Policy Agent that generates multi-turn CUDA/Triton kernels conditioned on selected skills, and a Skill Summary Agent that distills successful rollouts into reusable skills. Candidate skills are added only after execution-based verification confirms reproducible speedups. All three agents share a single LLM backbone, are initialized via a structured SFT cold start on diversity-filtered data, and are then jointly optimized end-to-end with multi-turn REINFORCE and per-agent advantage estimation. On KernelBench, daVinci-kernel-14B achieves 37.2%, 70.6%, and 32.2% on Level 1, Level 2, and Level 3 under the Fast$_1$ threshold, outperforming the strongest prior RL-trained model, Dr.Kernel-14B.

Problem

• GPU kernel optimization
• execution efficiency
• reinforcement learning
• skill discovery
• automatic code generation

Innovation

• reinforcement learning
• GPU kernel optimization
• skill library co-evolution
• large language model
• code generation