Optimal Learning Under Tsybakov Noise

2026-08-08

AI Summary

This work resolves a two-decade-old open problem in PAC learning under the Tsybakov noise model by closing the long-standing logarithmic gap between known upper and lower bounds. The authors introduce an adaptive partitioning strategy that groups regions of the instance space according to their local noise levels and constructs hypotheses tailored to satisfy localized error constraints within each region. By integrating this approach with recent advances in agnostic learning, they achieve optimal learning guarantees for general concept classes. This method yields the first upper bound that exactly matches the established lower bound, thereby establishing tight error rates and attaining theoretically optimal learning performance under Tsybakov noise.

Abstract

Probably Approximately Correct (PAC) learning [Val84] is a fundamental learning model that has been extensively investigated. In this model, $\mathcal{H} \subseteq \{0,1\}^{\mathcal{X}}$ is a concept class, and $h^*\in\mathcal{H}$ is the target concept to be learned. Having access to i.i.d. labeled examples from a distribution $\mathcal{D}$ over $\mathcal{X}\times\{0,1\}$, which admits $h^*$ as the best concept in $\mathcal{H}$, the goal is to design a learning algorithm that outputs a hypothesis having low error competitive to $h^{*}$ with high probability. This model was initially studied under the realizable setting, which assumes that $h^*$ has no error. A natural relaxation is to allow label noise, that is, the true label can be flipped with probability $η\in(0,1/2)$. In reality, certain labels might be extremely noisy, especially for those points near the decision boundary. Hence, it is natural to allow very noisy points, though only rarely. This is quantified by a noise model introduced by [MT99] and [Tsy04], now known as Tsybakov noise. For learning general concept classes, [MN06] gave the general upper and lower bounds for error guarantees under Tsybakov noise. However, their upper and lower bounds differ by a logarithmic factor. Resolving this gap has remained a well-known open question for the past twenty years. In this work, we resolve this open question by improving the upper bound to match the best known lower bound, thus establishing the optimal error guarantee for learning under Tsybakov noise. Our learning algorithm operates by adaptively partitioning the instance space into regions, roughly corresponding to different noise levels, and returning a hypothesis in the concept class satisfying a specific error constraint for each region. Our technique shares a conceptual foundation with several recent advances in non-realizable learning, such as [HLZ24] and [Han25].

Problem

• Tsybakov noise
• PAC learning
• optimal error guarantee
• learning theory
• noise model

Innovation

• Tsybakov noise
• PAC learning
• optimal error guarantee
• adaptive partitioning
• non-realizable learning