Focus, Align, and Sustain: Counteracting Gradient Dilution in Incremental Object Detection

2026-06-13

AI Summary

This work addresses catastrophic forgetting in incremental object detection with detection Transformers, where sequential learning induces gradient dilution that degrades performance on previously learned classes. The study is the first to identify a tripartite mechanism underlying gradient dilution—namely, signal dispersion, assignment drift, and support collapse—and proposes the FAS framework to systematically mitigate these issues. FAS harmonizes gradient flow through three complementary strategies: injecting priors into queries to concentrate discriminative signals, aligning query–target assignments via deterministic anchor distillation, and preserving the feature manifold of old classes through structured replay. Evaluated under a challenging 40+10×4 incremental setting, FAS substantially enhances optimization stability and outperforms state-of-the-art methods by over 5.0 AP.

Abstract

Adapting Detection Transformers to Incremental Object Detection (IOD) poses a systemic challenge, as set-based optimization is inherently destabilized by sequential learning. In this work, we identify Gradient Dilution as the root cause of performance degradation, wherein optimization signals required to preserve old knowledge are progressively weakened. This phenomenon manifests as a cascading erosion of preservation gradients in magnitude, direction, and support coverage, driven by three tightly coupled factors: Signal Dispersion, where foreground gradients are overwhelmed by background noise; Assignment Drift, where stochastic query-target matching induces inconsistent gradient trajectories; and Support Attrition, where gradients from retained samples insufficiently cover the old-class feature space, weakening decision boundaries under interference from new classes. To counteract this, we propose FAS, a unified framework that Focuses, Aligns, and Sustains gradient flow throughout incremental learning. Specifically, we introduce prior-injected queries to focus discriminative signals by filtering background interference at the source. We further propose deterministic anchor distillation to align query-target assignments and enforce semantic consistency across stages under unstable matching. Finally, we devise manifold-support replay to sustain distributional support of old classes, counteracting representational erosion induced by continual updates. Extensive experiments show that FAS restores robust optimization dynamics and outperforms state-of-the-art methods, achieving over 5.0 AP improvement in the challenging 40+10x4 incremental setting.

Problem

• Incremental Object Detection
• Gradient Dilution
• Detection Transformers
• Catastrophic Forgetting
• Continual Learning

Innovation

• Gradient Dilution
• Incremental Object Detection
• Detection Transformers
• Manifold-Support Replay
• Deterministic Anchor Distillation