Targeted Testing of Compiler Optimizations via Grammar-Level Composition Styles

2025-12-03

AI Summary

Compiler optimization fuzzing faces two key bottlenecks: (1) fixed optimization sequences fail to capture inter-stage interactions, and (2) syntactically sensitive trigger conditions are rarely generated via conventional mutation. This paper proposes a syntax-guided directed fuzzing method for compiler optimizations, introducing the novel concept of “compositional style”—leveraging structural program relationships (e.g., nesting, adjacency) to guide mutations and precisely activate optimization logic missed by traditional pipelines. Our approach employs syntax-driven automatic operator synthesis and crossover strategies, enabling rapid adaptation to new languages and modular IRs such as MLIR. Evaluations on LLVM and MLIR demonstrate a 2.6–2.8× increase in optimization trigger count, an 8–11% improvement in code coverage, and the discovery of 12 previously undetected LLVM optimization bugs—each missed by standard optimization pipelines.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Ensuring the correctness of compiler optimizations is critical, but existing fuzzers struggle to test optimizations effectively. First, most fuzzers use optimization pipelines (heuristics-based, fixed sequences of passes) as their harness. The phase-ordering problem can enable or preempt transformations, so pipelines inevitably miss optimization interactions; moreover, many optimizations are not scheduled, even at aggressive levels. Second, optimizations typically fire only when inputs satisfy specific structural relationships, which existing generators and mutations struggle to produce. We propose targeted fuzzing of individual optimizations to complement pipeline-based testing. Our key idea is to exploit composition styles - structural relations over program constructs (adjacency, nesting, repetition, ordering) - that optimizations look for. We build a general-purpose, grammar-based mutational fuzzer, TargetFuzz, that (i) mines composition styles from an optimization-relevant corpus, then (ii) rebuilds them inside different contexts offered by a larger, generic corpus via synthesized mutations to test variations of optimization logic. TargetFuzz is adaptable to a new programming language by lightweight, grammar-based, construct annotations - and it automatically synthesizes mutators and crossovers to rebuild composition styles. No need for hand-coded generators or language-specific mutators, which is particularly useful for modular frameworks such as MLIR, whose dialect-based, rapidly evolving ecosystem makes optimizations difficult to fuzz. Our evaluation on LLVM and MLIR shows that TargetFuzz improves coverage by 8% and 11% and triggers optimizations 2.8$ imes$ and 2.6$ imes$, compared to baseline fuzzers under the targeted fuzzing mode. We show that targeted fuzzing is complementary: it effectively tests all 37 sampled LLVM optimizations, while pipeline-fuzzing missed 12.

Problem

• Targeted fuzzing tests individual compiler optimizations effectively
• It addresses limitations of pipeline-based testing missing optimization interactions
• Generates specific program structures to trigger optimization conditions

Innovation

• Targeted fuzzing tests individual compiler optimizations directly
• Uses grammar-based mutations to rebuild optimization-specific composition styles
• Automatically synthesizes mutators from annotated language grammars