LEO: Tracing GPU Stall Root Causes via Cross-Vendor Backward Slicing

2026-04-21

AI Summary

This work addresses the limitation of existing GPU performance analysis tools, which can identify stall locations but fail to uncover the root causes of stalls across diverse vendor architectures. To bridge this gap, we propose the first unified root-cause analysis framework supporting NVIDIA, AMD, and Intel GPUs. Our approach performs cross-vendor backward slicing from stalled instructions, integrating register dependency analysis with vendor-specific synchronization modeling to precisely trace and attribute performance bottlenecks. Evaluated on 21 real-world workloads, the method achieves a geometric mean speedup of 1.73–1.82×, demonstrating both effectiveness and generalizability. Furthermore, it reveals that identical kernels require architecture-specific optimizations across different GPU vendors, significantly enhancing the efficacy of large-model-assisted optimization.

Technology Category

• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Solvers and Tools
• Search and Optimization: Evaluation and Analysis

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

More than half of the Top 500 supercomputers employ GPUs as accelerators. On GPU-accelerated platforms, developers face a key diagnostic gap: profilers show source lines where stalls occur, but not why they occur. Furthermore, the same kernel may have different stalls and underlying causes on different GPUs. This paper presents LEO, a root-cause analyzer for NVIDIA, AMD, and Intel GPUs that performs backward slicing from stalled instructions, considering dependencies arising from registers as well as vendor-specific synchronization mechanisms. LEO attributes GPU stalls to source instructions with the goal of explaining root causes of these inefficiencies. Across 21 workloads on three GPU platforms, LEO-guided optimizations deliver geometric-mean speedups of 1.73$\times$--1.82$\times$. Our case studies show that (1) the same kernel may require different optimizations for different GPU architectures, and (2) LEO's structured diagnostics improve code optimization with large language models relative to code-only and raw-stall-count baselines.

Problem

• GPU stall
• root cause analysis
• cross-vendor
• performance diagnosis
• backward slicing

Innovation

• backward slicing
• GPU stall analysis
• cross-vendor
• root-cause diagnosis
• performance optimization