Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Experts

2025-07-09

AI Summary

To address the dual challenges of intra-domain class imbalance and cross-domain distribution shift in Domain-Incremental Learning (DIL), this paper proposes the Dual-Balanced Collaborative Experts (DBCE) framework. DBCE employs frequency-aware expert grouping and dynamic expert selection to enable specialized modeling of classes with varying frequencies. It further introduces a balanced Gaussian sampling strategy for pseudo-feature generation, leveraging historical statistics to mitigate underlearning of minority classes and cross-domain knowledge degradation. A dedicated loss function is designed to jointly optimize knowledge retention and class balance. Evaluated on four benchmark datasets, DBCE significantly improves minority-class accuracy while effectively suppressing catastrophic forgetting. It achieves dynamic performance equilibrium between majority and minority classes across both old and new domains, thereby enhancing the robustness and adaptability of DIL in realistic imbalanced scenarios.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Domain-Incremental Learning (DIL) focuses on continual learning in non-stationary environments, requiring models to adjust to evolving domains while preserving historical knowledge. DIL faces two critical challenges in the context of imbalanced data: intra-domain class imbalance and cross-domain class distribution shifts. These challenges significantly hinder model performance, as intra-domain imbalance leads to underfitting of few-shot classes, while cross-domain shifts require maintaining well-learned many-shot classes and transferring knowledge to improve few-shot class performance in old domains. To overcome these challenges, we introduce the Dual-Balance Collaborative Experts (DCE) framework. DCE employs a frequency-aware expert group, where each expert is guided by specialized loss functions to learn features for specific frequency groups, effectively addressing intra-domain class imbalance. Subsequently, a dynamic expert selector is learned by synthesizing pseudo-features through balanced Gaussian sampling from historical class statistics. This mechanism navigates the trade-off between preserving many-shot knowledge of previous domains and leveraging new data to improve few-shot class performance in earlier tasks. Extensive experimental results on four benchmark datasets demonstrate DCE's state-of-the-art performance.

Problem

• Address intra-domain class imbalance in continual learning
• Mitigate cross-domain class distribution shifts effectively
• Balance knowledge preservation and new data utilization

Innovation

• Frequency-aware expert group for intra-domain imbalance
• Dynamic expert selector with balanced Gaussian sampling
• Dual-balance framework for cross-domain knowledge transfer