Numerical Superoptimization for Library Learning

2026-03-25

AI Summary

This work proposes GrowLibm, a novel approach for automatically discovering high-value mathematical primitives tailored to floating-point workloads. By repurposing the numerical superoptimizer Herbie as a library learning tool, the method systematically mines, ranks, and deduplicates candidate primitives, leveraging for the first time in adaptive numerical library expansion the search, equivalence reasoning, and cost modeling mechanisms from superoptimization. The approach integrates counterfactual utility evaluation, equivalence generalization, and redundancy pruning, and deploys end-to-end via an LLVM IR matcher. Evaluated on real-world applications including PROJ, CoolProp, and Basilisk, GrowLibm identifies compact, reusable primitives that, when integrated, yield up to 2.2× kernel speedups, improve accuracy from 56.0% to 93.5%, and deliver up to 5% end-to-end performance gains.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Other Foundations of Machine Learning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Numerical software depends on fast, accurate implementations of mathematical primitives like sin, exp, and log. Modern superoptimizers can optimize floating-point kernels against a given set of such primitives, but a more fundamental question remains open: which new primitives are worth implementing in the first place? We formulate this as numerical library learning: given a workload of floating-point kernels, identify the mathematical primitives whose expert implementations would most improve speed and accuracy. Our key insight is that numerical superoptimizers already have the machinery well-suited to this problem. Their search procedures happen to enumerate candidate primitives, their equivalence procedures can generalize and deduplicate candidates, and their cost models can estimate counterfactual utility: how much the workload would improve if a given primitive were available. We present GrowLibm, which repurposes the Herbie superoptimizer as a numerical library learner. GrowLibm mines candidate primitives from the superoptimizer's intermediate search results, ranks them by counterfactual utility, and prunes redundant candidates. Across three scientific applications (PROJ, CoolProp, and Basilisk), GrowLibm identifies compact, reusable primitives that can be implemented effectively using standard numerical techniques. When Herbie is extended with these expert implementations, kernel speed improves by up to 2.2x at fixed accuracy, and maximum achievable accuracy also improves, in one case from 56.0% to 93.5%. We also prototype an LLVM matcher that recognizes learned primitives in optimized IR, recovering 26 replacement sites across five PROJ projections and improving end-to-end application performance by up to 5%.

Problem

• numerical library learning
• mathematical primitives
• floating-point kernels
• superoptimization
• library synthesis

Innovation

• numerical superoptimization
• library learning
• mathematical primitives
• counterfactual utility
• GrowLibm