REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling

2026-07-03

AI Summary

This work addresses the challenge of catastrophic forgetting in open-world object detection, where storing raw training data is infeasible. It introduces the first rehearsal-free incremental learning framework for this setting, leveraging low-rank adapters (LoRA) to decouple general and task-specific knowledge. The proposed method incorporates a dual-stage objectness modeling (DSOM) mechanism combined with a calibrated Gaussian negative log-likelihood (CG-NLL) distance metric to effectively discriminate between known and unknown object categories. Experimental results demonstrate that the approach surpasses existing replay-based methods in both detection accuracy and unknown class discovery, establishing a new state-of-the-art performance and setting a strong benchmark for rehearsal-free open-world object detection.

Abstract

Open-World Object Detection (OWOD) requires detectors to identify previously unseen objects as unknown and incrementally incorporate them into the set of known categories, while preserving previously acquired knowledge. Existing frameworks rely heavily on exemplar replay to mitigate catastrophic forgetting, but in some real applications, storing raw data conflicts with data access restrictions and leads to data exposure risks, while incurring significant memory overhead. In this paper, we propose REAL-OW, a novel rehearsal-free framework that decouples incremental knowledge through a collaborative adapter architecture based on Low-Rank Adaptation (LoRA). Specifically, we deploy General Adapters (GAs) in the backbone to enable the significance-aware refinement of cross-task universal representations, while Specific Adapters (SAs) in the decoder provide orthogonal storage for task-specific expertise. To resolve representation drift in objectness modeling under rehearsal-free constraints, we introduce Dual-Stage Objectness Modeling (DSOM), which alternates between feature aggregation and boundary consolidation to stabilize objectness distributions while maintaining the separation between known and unknown categories. Furthermore, DSOM is supported by a Calibrated Gaussian Negative Log-Likelihood (CG-NLL) distance tailored for the dispersed feature distributions inherent in rehearsal-free settings. Extensive evaluations demonstrate that REAL-OW achieves state-of-the-art performance, surpassing existing exemplar replay methods in both detection precision and unknown discovery. Our approach establishes a new baseline for rehearsal-free OWOD.

Problem

• Open-World Object Detection
• Catastrophic Forgetting
• Rehearsal-Free Learning
• Incremental Learning
• Unknown Object Discovery

Innovation

• Low-Rank Adaptation
• Rehearsal-Free Learning
• Open-World Object Detection
• Dual-Stage Objectness Modeling
• Calibrated Gaussian NLL