Equivalence of Coarse and Fine-Grained Models for Learning with Distribution Shift

2026-05-07

AI Summary

This work investigates the relationship between coarse-grained (TDS) and fine-grained (PQ) learning models under distribution-independent settings, with a focus on the learnability of halfspaces. By establishing a black-box reduction, it proves for the first time the equivalence of TDS and PQ learnability over arbitrary distributions, which implies the computational hardness of learning halfspaces in the TDS framework. To circumvent this barrier, the paper introduces an efficient PQ learning algorithm that combines Forster transformation with membership queries and enhances TDS learners’ discriminative power via branching programs. The contributions include a general reduction between the two learning models, a characterization of the inherent difficulty of TDS learning for halfspaces, and a demonstration of the pivotal role of membership queries in overcoming this hardness.

Abstract

Recent work on provably efficient algorithms for learning with distribution shift has focused on two models: PQ learning (Goldwasser et al. (2020)) and TDS learning (Klivans et al. (2024)). Algorithms for TDS learning are allowed to reject a test set entirely if distribution shift is detected. In contrast, PQ learners may only reject points that are deemed out-of-distribution on an individual basis. Our main result is a surprising equivalence between these two models in the distribution-free setting. In particular, we give an efficient black-box reduction from PQ learning to TDS learning for any Boolean concept class. This equivalence implies the first hardness results for distribution-free TDS learning of basic classes such as halfspaces. The main technical contribution underlying our equivalence is a method for boosting, via branching programs, the weak distinguishing power of TDS learners that have rejected the target domain. We also show that giving a learner access to membership queries sidesteps these hardness results and allows for efficient, distribution-free PQ learnability of halfspaces. Our algorithm iteratively recovers large-margin separators obtained by applying successive Forster transforms on the training data.

Problem

• distribution shift
• PQ learning
• TDS learning
• distribution-free learning
• halfspaces

Innovation

• distribution shift
• PQ learning
• TDS learning
• black-box reduction
• membership queries