A Mechanism-Driven Theory of Phase Transitions in Active Learning

2026-06-30

AI Summary

This work addresses the lack of a general explanation for performance dynamics under varying labeling budgets in active learning, where conventional stage划分 based on fixed label counts fails to generalize. The authors propose a mechanism-driven phase transition theory that reinterprets the budgeting process as shifts in the dominant generalization mechanisms, identifying three distinct phases: data-driven, transitional, and model-driven. By integrating PAC risk decomposition, dynamic reconstruction, measurable proxy metrics, and piecewise regression analysis, they establish a quantifiable framework for phase identification. The study reveals that the efficiency of an acquisition strategy hinges on the alignment between its inductive bias and the prevailing generalization bottleneck. Experiments on natural and medical image datasets validate the three-phase model and demonstrate that self-supervised representations can advance phase transitions, offering a unified foundation for designing phase-aware active learning algorithms.

Abstract

Active learning (AL) performance is known to be budget-dependent, yet regimes are typically defined by heuristic label counts that fail to generalize across datasets or architectures. We characterize AL dynamics by reframing budget regimes as shifts in the dominant generalization mechanism. By reinterpreting PAC-style risk components as dynamic interacting terms, we prove that dominance shifts are structurally unavoidable, creating a moving bottleneck for generalization. We operationalize this using measurable proxies and a segmented regression procedure to identify a tripartite taxonomy: data-driven, transition, and model-driven phases. Our framework explains the long-standing observation that representativeness, coverage, and uncertainty strategies excel at different stages. Experiments across natural and medical imaging show that AL efficiency depends on the alignment between the strategy's inductive bias and the active bottleneck. Moreover, self-supervised representation shift transitions earlier along the labeling trajectory, highlighting the role of representation quality in shaping AL dynamics. Overall, this work provides a unified framework for the next generation of transition-aware AL algorithms.

Problem

• active learning
• phase transitions
• generalization mechanisms
• budget regimes
• inductive bias

Innovation

• active learning
• phase transitions
• generalization mechanisms
• self-supervised representation
• inductive bias