Numerical Fuzz: A Type System for Rounding Error Analysis

2024-05-07

Proc. ACM Program. Lang.

AI Summary

This work addresses the challenge of statically quantifying rounding errors in floating-point computations. We introduce Λnum, a functional language that—uniquely—integrates sensitivity analysis with graded monads within a linear type system, enabling fully automatic, sound static inference of upper bounds on rounding errors in numerical programs. Λnum natively models IEEE 754 rounding semantics and supports extensions to nondeterministic and stochastic rounding. By rigorously connecting denotational and operational semantics, we establish, for the first time at the type level, soundness guarantees for inferred error bounds. Our prototype implementation demonstrates effectiveness across multiple classical numerical algorithms: it achieves error-bound precision comparable to state-of-the-art tools while significantly accelerating inference speed.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Algorithms operating on real numbers are implemented as floating-point computations in practice, but floating-point operations introduce roundoff errors that can degrade the accuracy of the result. We propose Λnum, a functional programming language with a type system that can express quantitative bounds on roundoff error. Our type system combines a sensitivity analysis, enforced through a linear typing discipline, with a novel graded monad to track the accumulation of roundoff errors. We prove that our type system is sound by relating the denotational semantics of our language to the exact and floating-point operational semantics. To demonstrate our system, we instantiate Λnum with error metrics proposed in the numerical analysis literature and we show how to incorporate rounding operations that faithfully model aspects of the IEEE 754 floating-point standard. To show that Λnum can be a useful tool for automated error analysis, we develop a prototype implementation for Λnum that infers error bounds that are competitive with existing tools, while often running significantly faster. Finally, we consider semantic extensions of our graded monad to bound error under more complex rounding behaviors, such as non-deterministic and randomized rounding.

Problem

• Analyzing roundoff errors in floating-point computations.
• Developing a type system for quantitative error bounds.
• Automating error analysis with competitive performance.

Innovation

• Functional language with roundoff error type system
• Combines sensitivity analysis and graded monad
• Prototype infers competitive error bounds fast