Neyman-Pearson Classification under Both Null and Alternative Distributions Shift

2025-11-10

AI Summary

This paper addresses the Neyman–Pearson (NP) classification problem in transfer learning under *simultaneous shift* of both the null distribution μ₀ and the alternative distribution μ₁: minimizing the type-II error under μ₁ while constraining the type-I error under μ₀ to be at most a pre-specified level α. Existing NP transfer methods handle only *unilateral* distributional shift, rendering them inadequate for real-world scenarios where both source-domain distributions shift. To bridge this gap, we propose an adaptive NP transfer learning framework that jointly leverages statistical hypothesis testing and data-driven weighting to selectively exploit source-domain information—significantly reducing both error types when the source is beneficial, and automatically avoiding negative transfer when it is irrelevant or harmful. Our method is the first to provide provably statistically optimal guarantees under *dual-distribution shift*, combining theoretical rigor with computational efficiency.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Learning & Optimization for NLP
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

We consider the problem of transfer learning in Neyman-Pearson classification, where the objective is to minimize the error w.r.t. a distribution $mu_1$, subject to the constraint that the error w.r.t. a distribution $mu_0$ remains below a prescribed threshold. While transfer learning has been extensively studied in traditional classification, transfer learning in imbalanced classification such as Neyman-Pearson classification has received much less attention. This setting poses unique challenges, as both types of errors must be simultaneously controlled. Existing works address only the case of distribution shift in $mu_1$, whereas in many practical scenarios shifts may occur in both $mu_0$ and $mu_1$. We derive an adaptive procedure that not only guarantees improved Type-I and Type-II errors when the source is informative, but also automatically adapt to situations where the source is uninformative, thereby avoiding negative transfer. In addition to such statistical guarantees, the procedures is efficient, as shown via complementary computational guarantees.

Problem

• Transfer learning for Neyman-Pearson classification with distribution shifts
• Simultaneously controlling Type-I and Type-II errors under shifted distributions
• Adaptive procedure preventing negative transfer from uninformative source data

Innovation

• Adaptive transfer learning for Neyman-Pearson classification
• Handles distribution shifts in both null and alternative hypotheses
• Prevents negative transfer with statistical guarantees