A Fine-Grained Understanding of Uniform Convergence for Halfspaces

2026-05-07

AI Summary

This work investigates the fine-grained behavior of uniform convergence in halfspace learning, going beyond the classical worst-case VC-dimension bounds. By distinguishing between homogeneous and inhomogeneous halfspaces and employing a combination of risk-band decomposition and localized critical wedge analysis, the study provides a systematic characterization of generalization error in both realizable and agnostic settings. The main contributions include the first proof that homogeneous halfspaces in two dimensions achieve an $O(1/n)$ convergence rate in the realizable case; in the agnostic setting, it establishes deviation bounds over risk bands without logarithmic factors and demonstrates the unavoidable $\ln\ln n$ cost of union bounds. Furthermore, it shows that the VC bound is essentially tight for inhomogeneous halfspaces, whereas homogeneous halfspaces admit significantly sharper rates, with matching upper and lower bounds that fully delineate the refined landscape of uniform convergence for halfspaces.

Abstract

We study the fine-grained uniform convergence behavior of halfspaces beyond worst-case VC bounds. For inhomogeneous halfspaces in $\mathbb{R}^d$ with $d\ge 2$, we show that standard first-order VC bounds are essentially tight: even consistent hypotheses can incur population error $Θ(d\ln(n/d)/n)$, and in the agnostic setting the deviation scales as $\sqrt{τ\ln(1/τ)}$ at true error $τ$. In contrast, homogeneous halfspaces in $\mathbb{R}^2$ exhibit a markedly different behavior. In the realizable case, every hypothesis consistent with the sample has error $O(1/n)$. In the agnostic case, we prove a bandwise, log-free deviation bound on each dyadic risk band via a critical-wedge localization argument. Unioning over bands incurs only a $\ln\ln n$ overhead, and we establish a matching lower bound showing this overhead is unavoidable. Together, these results give a fine-grained and nearly complete picture of uniform convergence for halfspaces, revealing sharp dimensional and structural thresholds.

Problem

• uniform convergence
• halfspaces
• VC bounds
• agnostic learning
• realizable case

Innovation

• uniform convergence
• halfspaces
• VC bounds
• critical-wedge localization
• dyadic risk bands