Search papers, labs, and topics across Lattice.
This paper investigates the learning dynamics of logits debiasing in long-tailed semi-supervised learning (LTSSL), revealing that class-imbalance bias significantly influences predictions. By deriving a step-wise decomposition of logits updates, the authors demonstrate that predictions converge to class priors, which can be mitigated through methods like logit adjustment and reweighting. They introduce DyTrim, a dynamic pruning framework that reallocates gradient budgets effectively, achieving improved generalization and reduced class bias across various LTSSL benchmarks.
Class-imbalance bias in long-tailed semi-supervised learning can be systematically mitigated with a novel dynamic pruning approach that reallocates gradient budgets for better generalization.
Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes, leading to degraded generalization. While many long-tailed semi-supervised learning (LTSSL) methods have been proposed, the mechanisms by which they implicitly debias logits remain poorly understood. In this work, we revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing. Specifically, we derive a step-wise decomposition of the logits updates, showing that predictions are dominated by class-imbalance bias that reliably reflects label priors. To expose this effect, we use the logits of a task-irrelevant baseline image as an indicator of accumulated bias and prove that they converge to the class prior. This provides a unified view where LTSSL remedies such as logit adjustment, reweighting, and resampling correspond to reshaping gradient dynamics. Based on this insight, we propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data. We provide theoretical guarantees that DyTrim reduces class bias and improves generalization. Extensive experiments on standard LTSSL benchmarks show consistent gains across architectures and methods. Code available at: https://jiajun0425.github.io/DyTrim
