Data-driven Test Generation for Fuzzing AI Compiler

2026-01-24

AI Summary

This work addresses the critical challenge of ensuring correctness and reliability in AI compilers, which are prone to introducing bugs during cross-hardware deployment. To this end, we propose the first end-to-end, phase-aware fuzzing framework for AI compilers, systematically targeting three key compilation stages: model loading, high-level optimization, and low-level optimization. Our approach integrates three tailored strategies—library-level test migration (OPERAs), optimization-aware computation graph synthesis (OATest), and low-level IR seed mutation (HARMONY)—to achieve comprehensive coverage across the entire compilation pipeline. Evaluated on four mainstream AI compilers, our framework uncovered 266 previously unknown bugs, significantly advancing both test coverage and defect detection capability in AI compiler testing.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Heuristic Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Artificial Intelligence (AI) compilers are critical for efficiently deploying AI models across diverse hardware platforms. However, they remain prone to bugs that can compromise both compiler reliability and model correctness. Thus, ensuring the quality of AI compilers is crucial. In this work, we present a unified data-driven testing framework that systematically addresses stage-specific challenges in AI compilers. Specifically, OPERA migrates tests for AI libraries to test various operator conversion logic in the model loading stage. OATest synthesizes diverse optimization-aware computational graphs for testing high-level optimizations. HARMONY generates and mutates diverse low-level IR seeds to generate hardware-optimization-aware tests for testing low-level optimizations. Together, these techniques provide a comprehensive, stage-aware framework that enhances testing coverage and effectiveness, detecting 266 previously unknown bugs in four widely used AI compilers.

Problem

• AI compiler
• compiler reliability
• model correctness
• bug detection
• testing

Innovation

• data-driven testing
• AI compiler fuzzing
• stage-aware test generation
• optimization-aware graph synthesis
• IR mutation