Improving Forward Compatibility in Class Incremental Learning by Increasing Representation Rank and Feature Richness

2024-03-22

Neural Networks

AI Summary

To address poor forward compatibility in class-incremental learning (CIL)—where learning new tasks degrades generalization performance on previously seen classes—this paper identifies representation rank as a critical, previously overlooked factor governing forward compatibility and proposes a rank-aware feature expansion mechanism. Without accessing old-task data, the method jointly leverages contrastive learning and low-rank decomposition to decouple features, while incorporating dynamic rank regularization and multi-granularity feature enrichment to cooperatively enhance both representation rank and feature diversity. Evaluated on standard benchmarks including CIFAR-100 and ImageNet-Subset, it achieves an average 8.2% improvement in forward compatibility accuracy while maintaining stable backward compatibility. The core contributions are: (i) uncovering the intrinsic relationship between representation rank and forward compatibility; and (ii) establishing the first rank-driven feature expansion framework for CIL that operates without replaying or storing historical data.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Problem

• Continual Learning
• Class Incremental Learning (CIL)
• Knowledge Retention

Innovation

• RFR method
• Class Incremental Learning (CIL)
• Knowledge Retention