Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection

2026-03-02

AI Summary

This work addresses the challenges of prompt degradation and supervision inconsistency in incremental object detection, where prompt coupling and drift often cause foreground instances of old classes to be mislabeled as background. To mitigate these issues, the authors propose Prompt Decoupling with Prototypes (PDP), a novel framework featuring a dual-pool architecture: a shared pool captures task-agnostic knowledge to support forward transfer, while a private pool learns task-specific features. Additionally, PDP introduces a prototype-guided pseudo-labeling mechanism that dynamically maintains a class prototype space, effectively suppressing prompt drift and ensuring consistent supervision. This approach establishes the first dual-pool prompt decoupling paradigm and achieves state-of-the-art performance, improving AP by 9.2% on MS-COCO and 3.3% on PASCAL VOC.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Computer Vision: Object Detection & Categorization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Incremental Object Detection (IOD) aims to continuously learn new object categories without forgetting previously learned ones. Recently, prompt-based methods have gained popularity for their replay-free design and parameter efficiency. However, due to prompt coupling and prompt drift, these methods often suffer from prompt degradation during continual adaptation. To address these issues, we propose a novel prompt-decoupled framework called PDP. PDP innovatively designs a dual-pool prompt decoupling paradigm, which consists of a shared pool used to capture task-general knowledge for forward transfer, and a private pool used to learn task-specific discriminative features. This paradigm explicitly separates task-general and task-specific prompts, preventing interference between prompts and mitigating prompt coupling. In addition, to counteract prompt drift resulting from inconsistent supervision where old foreground objects are treated as background in subsequent tasks, PDP introduces a Prototypical Pseudo-Label Generation (PPG) module. PPG can dynamically update the class prototype space during training and use the class prototypes to further filter valuable pseudo-labels, maintaining supervisory signal consistency throughout the incremental process. PDP achieves state-of-the-art performance on MS-COCO (with a 9.2\% AP improvement) and PASCAL VOC (with a 3.3\% AP improvement) benchmarks, highlighting its potential in balancing stability and plasticity. The code and dataset are released at: https://github.com/zyt95579/PDP\_IOD/tree/main

Problem

• Incremental Object Detection
• Prompt Degradation
• Prompt Coupling
• Prompt Drift

Innovation

• Prompt Decoupling
• Dual-pool Prompting
• Incremental Object Detection
• Prototypical Pseudo-Label Generation
• Prompt Drift Mitigation