DuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic

2025-06-26

AI Summary

Real-world object detection (e.g., autonomous driving, intelligent surveillance) demands simultaneous adaptation to both class-incremental and domain-incremental shifts—yet existing Class-Incremental Object Detection (CIOD) and Domain-Incremental Object Detection (DIOD) methods suffer from either poor cross-domain generalization or catastrophic forgetting, often relying on exemplar replay. This paper introduces Dual-Incremental Object Detection (DuIOD), a novel setting that unifies modeling of continual class and domain evolution without requiring stored samples. To address it, we propose the Dual-Incremental Task Arithmetic Fusion framework (DuET) and a direction-consistency loss to mitigate gradient sign conflicts; adopt a detector-agnostic model fusion strategy; and design the unified Robustness-Accuracy-Integration (RAI) metric, jointly evaluating forgetting mitigation and domain generalization. On Pascal Series and Diverse Weather Series, DuIOD achieves RAI improvements of 13.12% and 11.39%, with average retention rates of 89.3% and 88.57%, respectively—substantially outperforming state-of-the-art methods.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Real-world object detection systems, such as those in autonomous driving and surveillance, must continuously learn new object categories and simultaneously adapt to changing environmental conditions. Existing approaches, Class Incremental Object Detection (CIOD) and Domain Incremental Object Detection (DIOD) only address one aspect of this challenge. CIOD struggles in unseen domains, while DIOD suffers from catastrophic forgetting when learning new classes, limiting their real-world applicability. To overcome these limitations, we introduce Dual Incremental Object Detection (DuIOD), a more practical setting that simultaneously handles class and domain shifts in an exemplar-free manner. We propose DuET, a Task Arithmetic-based model merging framework that enables stable incremental learning while mitigating sign conflicts through a novel Directional Consistency Loss. Unlike prior methods, DuET is detector-agnostic, allowing models like YOLO11 and RT-DETR to function as real-time incremental object detectors. To comprehensively evaluate both retention and adaptation, we introduce the Retention-Adaptability Index (RAI), which combines the Average Retention Index (Avg RI) for catastrophic forgetting and the Average Generalization Index for domain adaptability into a common ground. Extensive experiments on the Pascal Series and Diverse Weather Series demonstrate DuET's effectiveness, achieving a +13.12% RAI improvement while preserving 89.3% Avg RI on the Pascal Series (4 tasks), as well as a +11.39% RAI improvement with 88.57% Avg RI on the Diverse Weather Series (3 tasks), outperforming existing methods.

Problem

• Simultaneously learning new object categories and adapting to environmental changes
• Overcoming catastrophic forgetting and domain shift in object detection
• Enabling real-time incremental learning for diverse object detectors

Innovation

• Dual Incremental Object Detection via Task Arithmetic
• Directional Consistency Loss mitigates sign conflicts
• Detector-agnostic framework supports YOLO and RT-DETR