Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolve

2026-01-28

AI Summary

This work addresses the limitations of traditional compilers, which rely on hand-crafted optimization heuristics that struggle to adapt to the growing complexity of modern software and hardware and incur high maintenance costs. The authors propose an intelligent agent framework that integrates large language models, evolutionary search, and automated tuning to form a closed-loop pipeline of generation, evaluation, and optimization. For the first time, this approach enables end-to-end synthesis of compact, executable C++ code embodying optimization decision logic without human intervention. Evaluated on LLVM’s function inlining and register allocation tasks, the automatically generated policies match or outperform expert-designed rules in both code size and performance. The study also provides preliminary evidence of the framework’s transferability to XLA.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Modern compilers rely on hand-crafted heuristics to guide optimization passes. These human-designed rules often struggle to adapt to the complexity of modern software and hardware and lead to high maintenance burden. To address this challenge, we present Magellan, an agentic framework that evolves the compiler pass itself by synthesizing executable C++ decision logic. Magellan couples an LLM coding agent with evolutionary search and autotuning in a closed loop of generation, evaluation on user-provided macro-benchmarks, and refinement, producing compact heuristics that integrate directly into existing compilers. Across several production optimization tasks, Magellan discovers policies that match or surpass expert baselines. In LLVM function inlining, Magellan synthesizes new heuristics that outperform decades of manual engineering for both binary-size reduction and end-to-end performance. In register allocation, it learns a concise priority rule for live-range processing that matches intricate human-designed policies on a large-scale workload. We also report preliminary results on XLA problems, demonstrating portability beyond LLVM with reduced engineering effort.

Problem

• compiler optimization
• heuristics
• manual engineering
• adaptability
• maintenance burden

Innovation

• compiler optimization
• evolutionary search
• large language model
• heuristic synthesis
• autotuning