Zero-Forgetting CISS via Dual-Phase Cognitive Cascades

2026-03-14

AI Summary

This work addresses the catastrophic forgetting in class-incremental semantic segmentation (CISS) caused by the Softmax classification head by proposing Cognitive Cascade Segmentation (CogCaS), a novel approach inspired by the two-stage nature of human cognition. CogCaS decouples the segmentation task into two distinct stages: category existence detection followed by category-specific segmentation. To the best of our knowledge, this is the first method to explicitly model CISS as a two-stage cascaded architecture, leveraging strict parameter isolation and task decoupling to fundamentally avoid knowledge overwriting and probability confusion. Theoretical analysis highlights the limitations of existing approaches, while extensive experiments demonstrate that CogCaS significantly outperforms state-of-the-art methods on PASCAL VOC 2012 and ADE20K, achieving near-zero forgetting particularly in long-sequence incremental settings.

Technology Category

• Computer Vision: Segmentation
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Continual semantic segmentation (CSS) is a cornerstone task in computer vision that enables a large number of downstream applications, but faces the catastrophic forgetting challenge. In conventional class-incremental semantic segmentation (CISS) frameworks using Softmax-based classification heads, catastrophic forgetting originates from Catastrophic forgetting and task affiliation probability. We formulate these problems and provide a theoretical analysis to more deeply understand the limitations in existing CISS methods, particularly Strict Parameter Isolation (SPI). To address these challenges, we follow a dual-phase intuition from human annotators, and introduce Cognitive Cascade Segmentation (CogCaS), a novel dual-phase cascade formulation for CSS tasks in the CISS setting. By decoupling the task into class-existence detection and class-specific segmentation, CogCaS enables more effective continual learning, preserving previously learned knowledge while incorporating new classes. Using two benchmark datasets PASCAL VOC 2012 and ADE20K, we have shown significant improvements in a variety of challenging scenarios, particularly those with long sequence of incremental tasks, when compared to exsiting state-of-the-art methods. Our code will be made publicly available upon paper acceptance.

Problem

• catastrophic forgetting
• class-incremental semantic segmentation
• continual learning
• task affiliation probability

Innovation

• Cognitive Cascade Segmentation
• Class-Incremental Semantic Segmentation
• Catastrophic Forgetting
• Dual-Phase Learning
• Strict Parameter Isolation