🤖 AI Summary
This work addresses the suppression of minority-class learning in deep neural networks under severe class imbalance, which arises from shared feature representations. To mitigate this issue, the authors propose a Class-Specific Branch Attention (CSBA) mechanism. By analyzing inter-layer gradient flows and constructing a gradient conflict matrix based on class-specific gradient cosine similarity, they reveal—through the lens of optimization dynamics—for the first time how majority classes dominate and suppress gradients of minority classes. A lightweight channel reweighting module is integrated into a multi-branch convolutional architecture to implicitly decouple features and gradients in a class-aware manner. Experiments demonstrate that the proposed method significantly improves minority-class performance without compromising overall accuracy: the F1 score for the Physical-Damage class increases from 0.261 to 0.522, and Macro-F1 on CIFAR-10-LT rises from 0.595 to 0.655.
📝 Abstract
Deep neural networks trained under severe class imbalance often exhibit degraded performance, typically attributed to statistical bias. In this work, we identify a complementary optimization-level pathology: inter-class gradient interference within shared representations, where gradients from majority classes suppress minority-class learning. To analyze this phenomenon, we introduce a diagnostic framework based on layer-wise gradient flow analysis and a Gradient Conflict Matrix, which quantifies interference using cosine similarity between class-specific gradients. Using this framework, we study multi-branch convolutional architectures and propose a lightweight modification, Class-Specific Branch Attention (CSBA), that enables branch-specific channel reweighting to reduce gradient coupling. This mechanism promotes implicit feature decoupling across branches while preserving architectural simplicity. Empirically, CSBA improves minority-class performance, increasing the F1 score for the Physical-Damage class from 0.261 to 0.522 under severe imbalance, while maintaining comparable overall accuracy. Validation on CIFAR-10-LT confirms that this behavior generalizes across imbalanced visual recognition settings, with Macro-F1 improving from 0.595 to 0.655. More broadly, our findings highlight the importance of considering optimization dynamics alongside statistical methods when designing architectures for imbalanced learning.