compute unified device architecture

This work addresses the challenge of automatically generating high-performance GPU tiled kernels from high-level tensor algebra expressions, thereby alleviating the burden of manual optimization. The authors propose an end-to-end compilation framework that integrates layer-wise lowering, expression rewriting, automated schedule search, reduction fusion, and tiling optimizations. For the first time, this framework automatically discovers high-efficiency kernels—comparable to FlashAttention-3—from the mathematical specification of attention operators, while introducing a novel scheduler that preserves program structural regularity. Evaluated on GH200 and RTX 5090 GPUs, the generated kernels achieve up to 23% and 42% higher throughput, respectively, and match or surpass hand-optimized cuDNN kernels across multiple long-sequence configurations.

This study addresses the GPU computational efficiency bottleneck in deep and machine learning. Methodologically, it proposes a task-aware GPU parallel architecture adaptation framework that systematically integrates CUDA stream-based concurrency, dynamic parallelism, and heterogeneous hardware (FPGA/TPU/ASIC) co-selection—implemented via deep integration into PyTorch, TensorFlow, and XGBoost. Its core contribution lies in establishing a transferable GPU optimization methodology, transcending model- or library-specific tuning. Experimental evaluation demonstrates 3–8× speedup across representative training and inference workloads. Furthermore, the authors open-source a modular, well-documented GPU optimization practice guide, substantially lowering the barrier to entry for AI practitioners seeking parallelization optimizations.

CUDA Graphs in PyTorch suffer from deployment challenges and high overhead—sometimes even yielding negative speedup—due to static graph constraints and redundant host-to-device parameter copies. This paper proposes a compiler-level, fully automatic optimization framework that requires no user code modification. First, it introduces a novel cost-benefit–driven dynamic graph selection mechanism that adaptively enables or bypasses graph capture based on runtime characteristics. Second, it eliminates redundant kernel parameter copying overhead—a previously unaddressed bottleneck. Third, it extends the scope of automatic static graph capture and reuse to support more complex ML workflows. The framework tightly integrates PyTorch 2’s compilation stack, CUDA Graphs’ hardware capabilities, DAG-structured program analysis, and runtime heuristic decision-making. Evaluated across diverse ML benchmarks, it consistently outperforms PyTorch 2, completely eliminates negative speedup, and delivers 1.3×–2.1× average end-to-end speedup.

This work addresses the lack of transparency in NVIDIA’s closed-source user-space driver, which obscures the translation of CUDA API calls into hardware commands and impedes understanding of GPU behavior and performance attribution. The authors propose a novel approach that requires no modification to the proprietary driver, instead leveraging an open-source kernel driver, memory-mapped path instrumentation, and hardware watchpoints on the GPU’s doorbell registers to capture and reconstruct the complete low-level command stream with unprecedented accuracy. This methodology reveals the true DMA patterns and performance characteristics of CUDA data transfers and demonstrates that the low overhead of CUDA Graphs stems from their streamlined and efficient command submission mechanism. By significantly enhancing the interpretability of GPU runtime behavior, this approach establishes a new paradigm for middleware analysis and hardware-software co-design.

SYCL’s cross-platform ecosystem lacks device-side dynamic memory management, hindering efficient heap allocation and garbage collection on heterogeneous accelerators. Method: This paper presents the first port of the CUDA-optimized memory manager Ouroboros to SYCL 2020, enabling device-side `malloc`/`free` and garbage collection across multiple backends—specifically CUDA and Intel DPC++. By abstracting hardware-specific details, reusing core memory management algorithms, and refactoring runtime semantics to be platform-agnostic, the framework supports compile-time backend selection and a unified memory interface. Contribution/Results: On NVIDIA GPUs, the implementation achieves near-native CUDA performance (<5% overhead); on Intel Arc GPUs, it demonstrates, for the first time in SYCL, functional device-side dynamic heap allocation with a measured throughput of 1.2 GB/s. This work fills a critical gap in SYCL’s real-time GPU memory management capabilities.

This work proposes KernelPro, a closed-loop multi-agent system designed to automatically generate high-performance and energy-efficient GPU kernel code. By integrating large language models with hardware micro-benchmarking tools, KernelPro employs semantic feedback operators, a two-tier tool-calling architecture, a domain-adapted Monte Carlo Tree Search (MCTS) strategy, and direct CuTe source-code generation to jointly optimize for both performance and energy efficiency. Evaluated on KernelBench, KernelPro achieves up to a 5.30× speedup over baseline implementations. Furthermore, when applied to expert-optimized Mixture-of-Experts (MoE) kernels, it outperforms hand-tuned Triton kernels by 1.23× in performance while reducing measured energy consumption by 11.6%.

This study addresses the lack of native GPU monitoring tools in R for computational biology, where users traditionally rely on external command-line utilities. To bridge this gap, we introduce CudaMon, an R package that leverages the NVIDIA Management Library (NVML) interface to provide real-time monitoring of GPU utilization, memory usage, temperature, and power consumption. CudaMon seamlessly integrates data export and visualization capabilities into standard R workflows, significantly enhancing the efficiency and reproducibility of GPU-accelerated analyses. In a large-scale single-cell RNA-seq analysis involving millions of brain cells, the tool successfully identified stages—such as PCA and UMAP—where GPU utilization exceeded 90%, while also uncovering performance bottlenecks during data management phases.

This work addresses the challenge of efficiently solving nonlinear optimization problems on GPUs, particularly in robotics and related domains. It introduces a novel approach that integrates symbolic programming with high-performance GPU computing. Leveraging the SymForce library, the system is the first to automatically compile symbolic expressions involving Lie group operations into customized CUDA kernels. An adaptive reordering strategy is further incorporated to enhance memory efficiency. Users need only define symbolic residual functions to trigger fully automated generation of highly optimized GPU code. Evaluated on bundle adjustment tasks using the BAL dataset, the proposed method achieves 5–20× speedups over the current state-of-the-art while reducing memory consumption and maintaining comparable numerical accuracy.

This work systematically evaluates the capability of large language models to generate expert-level, hardware-aware CUDA code, revealing a substantial gap between current model performance and human expert optimization. We introduce a multi-tiered benchmark encompassing kernel functions, operators, full applications, and open-ended challenge tasks, evaluated across Ampere, Hopper, and Blackwell GPU architectures. For the first time, end-to-end expert-level CUDA optimization is established as a formal evaluation criterion, combining GPU-specific microbenchmarks with domain-aware semantic validators. Experimental results demonstrate that while existing models—such as Claude-Opus-4.6 and GPT-5.4—can produce runnable code, they rarely reproduce critical optimization strategies employed by experts. Performance further degrades when application-level semantics are incorporated, underscoring fundamental limitations in current models’ hardware reasoning and tool integration capabilities.

This work addresses the lack of efficient support for dynamic migration of work items—such as rays—across GPUs in multi-node, multi-GPU data-parallel computing. The authors propose RaFI, a software framework built on CUDA and MPI, which introduces, for the first time, a unified interface enabling GPU kernels to succinctly forward work items to other GPUs while automatically managing the underlying communication and data transfers. By abstracting away the complexities of coordinated CUDA-MPI programming, RaFI significantly simplifies the development of multi-GPU collaborative applications. Empirical evaluation across several use cases demonstrates that the framework not only eases programming but also maintains high performance and strong scalability.