🤖 AI Summary
This paper addresses the problem of deriving upper bounds on the covering number for real algebraic varieties, images of polynomial maps, and semialgebraic sets in Euclidean space. Building upon the classical Yomdin–Comte estimate, we develop a unified, tight upper bound applicable to *nonsmooth* semialgebraic sets—not restricted to smooth varieties—via a significantly simplified proof. Our bound yields exact volume estimates for tubular neighborhoods, bridging a gap left by Lotz’s and Basu–Lerario’s methods, which fail for nonsmooth settings. The resulting theory provides a unifying framework for three key problems: (1) near-optimal covering bounds for low CP-rank tensors; (2) minimal sketching dimension characterization for randomized polynomial optimization; and (3) tight generalization error bounds for deep neural networks with ReLU or rational activation functions—some of which match or improve the best known results in the literature.
📝 Abstract
Covering numbers are a powerful tool used in the development of approximation algorithms, randomized dimension reduction methods, smoothed complexity analysis, and others. In this paper we prove upper bounds on the covering number of numerous sets in Euclidean space, namely real algebraic varieties, images of polynomial maps and semialgebraic sets in terms of the number of variables and degrees of the polynomials involved. The bounds remarkably improve the best known general bound by Yomdin-Comte, and our proof is much more straightforward. In particular, our result gives new bounds on the volume of the tubular neighborhood of the image of a polynomial map and a semialgebraic set, where results for varieties by Lotz and Basu-Lerario are not directly applicable. We illustrate the power of the result on three computational applications. Firstly, we derive a near-optimal bound on the covering number of low rank CP tensors, quantifying their approximation properties and filling in an important missing piece of theory for tensor dimension reduction and reconstruction. Secondly, we prove a bound on the required dimension for the randomized sketching of polynomial optimization problems, which controls how much computation can be saved through randomization without sacrificing solution quality. Finally, we deduce generalization error bounds for deep neural networks with rational or ReLU activation functions, improving or matching the best known results in the machine learning literature while helping to quantify the impact of architecture choice on generalization error.
