🤖 AI Summary
To mitigate catastrophic forgetting in continual learning, this paper proposes a lightweight replay mechanism that does not rely on generative models. Methodologically, it abandons GANs or VAEs and instead caches compact feature summaries of historical tasks, integrating task-adaptive feature distillation with gradient orthogonality constraints to explicitly preserve prior knowledge during parameter updates. This paradigm combines the knowledge recovery capability of generative replay with the training stability of exemplar-based replay methods. Evaluated on standard continual learning benchmarks—including CIFAR-100 and Tiny-ImageNet—the approach achieves an average accuracy gain of 5.2% over state-of-the-art methods such as iCaRL and LwF, while reducing memory overhead by 60%. The method significantly enhances model generalization and deployment efficiency in privacy-sensitive transfer learning scenarios, where data re-sampling or generation is prohibited.
