LLM-Based Porting of Optimized C++ to CUDA Through Deoptimization and Reoptimization

2026-06-04

AI Summary

This work addresses the challenge that CPU-oriented optimizations hinder large language models (LLMs) from directly generating efficient CUDA code. To overcome this, the authors propose a deoptimization–reoptimization (Deopt-Reopt) pipeline: first deoptimizing optimized C++ code to simplify its structure, then leveraging LLMs to translate and reoptimize it for GPUs. This study presents the first systematic exploration of deoptimization and reoptimization in LLM-driven CPU-to-GPU migration, integrating models such as O120 and Q235, both single-pass and iterative repair mechanisms, and a joint evaluation of performance and feasibility. Experiments demonstrate significant improvements in compilation success rates and performance on high-performance computing kernels—including conv2d, ddgemm, and bgemm—though effectiveness varies with kernel characteristics, model capabilities, and available search budgets.

Abstract

When porting high-performance computing (HPC) code from CPU to GPU, CPU-oriented optimizations may obstruct LLM-based CUDA translation. We design and evaluate a Deopt-Reopt workflow that first simplifies the input C++ code and then retranslates and reoptimizes it for CUDA, comparing it against direct translation (Direct) on twelve HPC kernels with two LLMs (gpt-oss-120b (O120) and qwen-3-235b-a22b-instruct-2507 (Q235)) in Single-shot (one pass) and Iterative (repeated refinement) settings. In Single-shot, among 18 testable cases Deopt-Reopt was significantly faster among successful trials (after BH-FDR correction) in five - most clearly for conv2d, where CPU- and GPU-oriented designs diverge - but Direct was faster in three, so removing CPU-specific optimizations is not universally beneficial. An exploratory Direct-3 control that equalizes the LLM-call count left Deopt-Reopt ahead in only four of nineteen testable cases, with Direct-3 ahead in four others. In Iterative, repeated generation and repair narrow the mode gap - markedly so for O120 - while Q235 retains large Deopt-Reopt advantages on conv2d, ddgemm, and bgemm. Deopt-Reopt's effect on feasibility is also mixed - sharply higher for some kernels Direct rarely compiles, lower for others. Because performance is conditioned on successful trials, the benefit is conditional rather than a guaranteed end-to-end gain. Overall, Deopt-Reopt is an effective but non-universal technique for LLM-based GPU porting, with gains that depend on the kernel, the model, the search budget, and the success rate.

Problem

• LLM-based porting
• CPU-to-GPU migration
• code optimization
• CUDA translation
• high-performance computing

Innovation

• Deopt-Reopt
• LLM-based code translation
• CUDA porting
• deoptimization
• GPU code optimization