Detecting Unknown Objects via Energy-based Separation for Open World Object Detection

2026-03-31

AI Summary

This work addresses the challenges of unknown object recognition and catastrophic forgetting in open-world object detection (OWOD) by proposing the DEUS framework. DEUS innovatively leverages equiangular tight frames (ETFs) to construct orthogonal subspaces that disentangle representations of known and unknown classes, and introduces an energy-based classifier for unsupervised unknown detection. To mitigate knowledge interference between old and new classes during incremental learning, it incorporates constrained memory replay and an enhanced knowledge distillation (EKD) loss. Evaluated on standard OWOD benchmarks, DEUS achieves substantial improvements in detecting unknown objects while preserving high accuracy on known categories.

Abstract

In this work, we tackle the problem of Open World Object Detection (OWOD). This challenging scenario requires the detector to incrementally learn to classify known objects without forgetting while identifying unknown objects without supervision. Previous OWOD methods have enhanced the unknown discovery process and employed memory replay to mitigate catastrophic forgetting. However, since existing methods heavily rely on the detector's known class predictions for detecting unknown objects, they struggle to effectively learn and recognize unknown object representations. Moreover, while memory replay mitigates forgetting of old classes, it often sacrifices the knowledge of newly learned classes. To resolve these limitations, we propose DEUS (Detecting Unknowns via energy-based Separation), a novel framework that addresses the challenges of Open World Object Detection. DEUS consists of Equiangular Tight Frame (ETF)-Subspace Unknown Separation (EUS) and an Energy-based Known Distinction (EKD) loss. EUS leverages ETF-based geometric properties to create orthogonal subspaces, enabling cleaner separation between known and unknown object representations. Unlike prior energy-based approaches that consider only the known space, EUS utilizes energies from both spaces to better capture distinct patterns of unknown objects. Furthermore, EKD loss enforces the separation between previous and current classifiers, thus minimizing knowledge interference between previous and newly learned classes during memory replay. We thoroughly validate DEUS on OWOD benchmarks, demonstrating outstanding performance improvements in unknown detection while maintaining competitive known class performance.

Problem

• Open World Object Detection
• Unknown Object Detection
• Catastrophic Forgetting
• Incremental Learning
• Unsupervised Discovery

Innovation

• Open World Object Detection
• Energy-based Separation
• Equiangular Tight Frame
• Unknown Object Discovery
• Memory Replay