Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning

2026-08-31

AI Summary

研究通过分析学习动态来解决长尾半监督学习中的类别不平衡偏差问题，提出DyTrim框架调整梯度分配以减少偏误。

Abstract

Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes, leading to degraded generalization. While many long-tailed semi-supervised learning (LTSSL) methods have been proposed, the mechanisms by which they implicitly debias logits remain poorly understood. In this work, we revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing. Specifically, we derive a step-wise decomposition of the logits updates, showing that predictions are dominated by class-imbalance bias that reliably reflects label priors. To expose this effect, we use the logits of a task-irrelevant baseline image as an indicator of accumulated bias and prove that they converge to the class prior. This provides a unified view where LTSSL remedies such as logit adjustment, reweighting, and resampling correspond to reshaping gradient dynamics. Based on this insight, we propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data. We provide theoretical guarantees that DyTrim reduces class bias and improves generalization. Extensive experiments on standard LTSSL benchmarks show consistent gains across architectures and methods. Code available at: https://jiajun0425.github.io/DyTrim

Problem

• long-tailed distributions
• semi-supervised learning
• pseudo-labels
• class-imbalance bias
• logits debiasing

Innovation

• logits debiasing
• learning dynamics
• DyTrim
• gradient budget reallocation