Floating-Point Neural Networks Are Provably Robust Universal Approximators

2025-06-19

AI Summary

This paper addresses the foundational question of whether neural networks satisfy the Interval Universal Approximation (IUA) theorem under floating-point arithmetic. For a finite-precision floating-point computation model, we provide the first rigorous proof that the IUA theorem holds: floating-point neural networks can exactly capture the direct image mapping of any target function after rounding, thereby overcoming the traditional reliance on real-number assumptions. Methodologically, our approach integrates abstract interpretation, interval semantics modeling, formalization of floating-point arithmetic, and straight-line program theory. Key contributions include: (i) establishing the unconditional expressive power of floating-point neural networks; (ii) providing a theoretical foundation for provable robustness; and (iii) proving that straight-line programs composed solely of floating-point addition and multiplication are computationally universal—capable of simulating all terminating floating-point programs.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

The classical universal approximation (UA) theorem for neural networks establishes mild conditions under which a feedforward neural network can approximate a continuous function $f$ with arbitrary accuracy. A recent result shows that neural networks also enjoy a more general interval universal approximation (IUA) theorem, in the sense that the abstract interpretation semantics of the network using the interval domain can approximate the direct image map of $f$ (i.e., the result of applying $f$ to a set of inputs) with arbitrary accuracy. These theorems, however, rest on the unrealistic assumption that the neural network computes over infinitely precise real numbers, whereas their software implementations in practice compute over finite-precision floating-point numbers. An open question is whether the IUA theorem still holds in the floating-point setting. This paper introduces the first IUA theorem for floating-point neural networks that proves their remarkable ability to perfectly capture the direct image map of any rounded target function $f$, showing no limits exist on their expressiveness. Our IUA theorem in the floating-point setting exhibits material differences from the real-valued setting, which reflects the fundamental distinctions between these two computational models. This theorem also implies surprising corollaries, which include (i) the existence of provably robust floating-point neural networks; and (ii) the computational completeness of the class of straight-line programs that use only floating-point additions and multiplications for the class of all floating-point programs that halt.

Problem

• Proving floating-point neural networks approximate functions accurately
• Comparing interval universal approximation in real vs floating-point settings
• Establishing robustness and computational completeness of floating-point networks

Innovation

• Floating-point neural networks approximate rounded functions perfectly
• First interval universal approximation theorem for floating-point networks
• Provably robust networks with floating-point operations