AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning

2026-06-18

AI Summary

This work addresses the challenge of leveraging large language models (LLMs) for compiler runtime performance tuning, which is hindered by microarchitectural complexity and measurement noise. The authors propose a novel multi-agent framework that, for the first time, exposes both compiler internal states and LLVM intermediate representation to the LLM, enabling it to jointly reason with runtime performance feedback and iteratively optimize compilation flags. This approach requires no model training and offers a plug-and-play, white-box alternative that overcomes the limitations of traditional black-box autotuning methods. Experimental results demonstrate geometric mean speedups of 1.043× and 1.117× over LLVM’s -O3 optimization level on x86-64 and ARM64 platforms, respectively, outperforming both expert-crafted heuristics and classical autotuning techniques.

Abstract

Large Language Models (LLMs) show promise for code compilation tasks, but applying them to runtime performance tuning is difficult due to complex microarchitectural effects and noisy runtime measurements. We present AutoPass, a multi-agent framework for compiler performance tuning that uses compiler and runtime evidence to guide LLM-generated optimization decisions. Rather than treating the compiler as a black box like prior auto-tuning schemes, AutoPass opens up the compiler to the LLM, enabling it to query compiler-internal optimization states and analyze the intermediate representation to orchestrate compiler options. The search process iteratively refines optimization configurations using measured runtime feedback to diagnose regressions and guide latency-improving edits. AutoPass operates in an inference-only, training-free setting and requires no offline training or task-specific fine-tuning, making it readily applicable to new benchmarks and platforms. We implement AutoPass on the LLVM compiler and evaluate it on server-grade x86-64 and embedded ARM64 systems. AutoPass outperforms expert-tuned heuristics and classical autotuning methods, achieving geometric-mean speedups of 1.043x and 1.117x over LLVM -O3 on x86-64 and ARM64, respectively.

Problem

• compiler performance tuning
• large language models
• runtime performance
• optimization
• microarchitectural effects

Innovation

• LLM agents
• compiler optimization
• evidence-guided tuning
• training-free inference
• intermediate representation analysis