Surprises in Proper Positive-Only Learning

2026-06-26

AI Summary

This study addresses the problem of proper learnability in binary classification when only positive examples are available, under the assumption that the hypothesis space coincides with the target concept class. By introducing a novel combinatorial property—uniform exterior separability—and integrating it with VC dimension, the work establishes the first complete characterization of proper learnability in this setting: it is achievable if and only if both finite VC dimension and uniform exterior separability hold. The analysis reveals fundamental separations between proper and improper learning, as well as between randomized and deterministic proper learning. It further demonstrates that empirical risk minimization (ERM) may entirely fail in this context and introduces several new combinatorial dimensions that underscore the profound differences between positive-sample-only learning and the standard PAC model.

Abstract

Binary classification from positive-only samples is a variant of PAC learning in which the learner receives i.i.d. samples from the positive region of an unknown target concept, but is evaluated under the original distribution (which places mass on both positive and negative regions). This model dates back to Natarajan [1987, STOC], and the characterization of improper learning is well-known -- it even appears in textbooks. The characterization of proper positive-only learning, however, has long remained open. In this work, we revisit and settle this question: a concept class is properly learnable from positive-only samples if and only if it has finite VC dimension and satisfies a new combinatorial condition, which we call uniform exterior separability. Together with several separation results, this characterization reveals a surprisingly rich landscape that differs sharply from standard PAC learning: proper and improper learning are separated, randomized and deterministic proper learning are separated, there are classes for which no ERM is a learner, and finite VC dimension does not suffice even for non-uniform learning. Along the way, we introduce new combinatorial dimensions that we believe can be of broader interest in learning theory.

Problem

• proper learning
• positive-only learning
• VC dimension
• combinatorial condition
• PAC learning

Innovation

• proper positive-only learning
• uniform exterior separability
• VC dimension
• combinatorial dimensions
• PAC learning