Teach YOLO to Remember: A Self-Distillation Approach for Continual Object Detection

2025-03-06

AI Summary

This work addresses catastrophic forgetting in YOLO-family anchor-free one-stage detectors under class-incremental learning. We propose YOLO-LwF, the first lightweight self-distillation framework for this setting. It integrates self-distillation—using the model’s own historical parameters as the teacher—with LwF-style knowledge distillation and experience replay, enabling stable knowledge transfer without external teacher models. Our key innovation is the first adaptation of self-distillation to the YOLO architecture, augmented by regression-aware feature alignment to mitigate cumulative localization noise during incremental training. Evaluated on PASCAL VOC and MS COCO benchmarks, YOLO-LwF achieves state-of-the-art performance, improving mAP by 2.1% and 2.9% over existing continual object detection methods, respectively.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Real-time object detectors like YOLO achieve exceptional performance when trained on large datasets for multiple epochs. However, in real-world scenarios where data arrives incrementally, neural networks suffer from catastrophic forgetting, leading to a loss of previously learned knowledge. To address this, prior research has explored strategies for Class Incremental Learning (CIL) in Continual Learning for Object Detection (CLOD), with most approaches focusing on two-stage object detectors. However, existing work suggests that Learning without Forgetting (LwF) may be ineffective for one-stage anchor-free detectors like YOLO due to noisy regression outputs, which risk transferring corrupted knowledge. In this work, we introduce YOLO LwF, a self-distillation approach tailored for YOLO-based continual object detection. We demonstrate that when coupled with a replay memory, YOLO LwF significantly mitigates forgetting. Compared to previous approaches, it achieves state-of-the-art performance, improving mAP by +2.1% and +2.9% on the VOC and COCO benchmarks, respectively.

Problem

• Mitigates catastrophic forgetting in YOLO-based object detection.
• Introduces self-distillation for continual learning in YOLO.
• Improves mAP on VOC and COCO benchmarks significantly.

Innovation

• Self-distillation approach for YOLO
• Replay memory mitigates catastrophic forgetting
• Improves mAP on VOC and COCO benchmarks