Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networks

2026-06-29

AI Summary

This study addresses the efficient approximation and learning of binary classification tasks defined by definable sets in o-minimal structures. Leveraging cell decomposition, the authors introduce traceable sets as surrogates for decision regions and combine ReLU neural network approximation with entropy estimation to establish the first quantitative approximation rates for such definable sets within the empirical risk minimization framework. The main contributions include achieving an approximation error of ε in the L^p norm using a network of size O(ε^{−p(n−1)/m}), and deriving a statistical learning rate for the expected misclassification error of order N^{−m/(m+pn−p)} based on N samples.

Abstract

We study binary classification problems whose decision sets are given by definable sets in o-minimal expansions of the real field. Motivated by cell decomposition of definable sets, we introduce traceable sets as a classical proxy for definable decision regions and analyze their approximation by ReLU neural networks. Under uniform bounds on the number of connected components and suitable $C^m$ extensions for the boundary functions, we prove that characteristic functions of traceable subsets of $[-1/2,1/2]^n$ can be approximated in $L^p$ to accuracy $\varepsilon>0$ by ReLU neural networks of size $\mathcal{O}(\varepsilon^{-p(n-1)/m})$, with depth independent of $\varepsilon$ and polynomially bounded weights. This establishes quantitative approximation rates for certain definable collections in o-minimal structures using ReLU neural networks. The same approach also yields the stated approximation rates for a subclass of definable maps $[-1/2,1/2]^n \to \mathbb{R}$. We then combine the approximation capabilities with entropy estimates for ReLU neural network classes to obtain statistical learning rates for empirical risk minimization with hinge loss. For $N$ uniformly distributed samples, the resulting classifiers achieve expected misclassification error of order $N^{-m/(m+pn-p)}$ up to an arbitrarily small polynomial loss.

Problem

• binary classification
• o-minimal structures
• ReLU neural networks
• definable sets
• approximation

Innovation

• o-minimal structures
• ReLU neural networks
• traceable sets
• approximation rates
• statistical learning rates