Continual Segmentation under Joint Nonstationarity

2026-05-19

AI Summary

This work addresses the non-stationarity challenge in semantic segmentation arising from the joint temporal shifts in class distribution, domain characteristics, and label availability. To tackle this, the authors propose a continual learning framework that integrates semi-supervised learning with parameter-level regularization. The approach balances model stability and plasticity through a gradient-adaptive stabilization mechanism and a prototype-anchored supervision strategy. Additionally, it enhances robustness under dynamic data streams by incorporating gradient-scaled stochastic perturbation regularization and a dual-verification pseudo-label filtering scheme based on confidence-prototype consistency. Experimental results demonstrate that the proposed method significantly outperforms existing approaches across diverse scenarios involving class-incremental, domain-incremental, and few-shot settings, thereby exposing the limitations of conventional continual segmentation models in jointly non-stationary environments.

Abstract

Evolving data streams induce joint nonstationarity in continual semantic segmentation, where semantic classes, input distributions, and supervision availability change simultaneously over time. This setting reflects practical structured prediction systems, yet remains largely unexplored in prior continual learning work, which typically studies these factors in isolation. We formalize continual segmentation under coupled class, domain, and label shifts and investigate learning in heterogeneous dense prediction environments with limited annotations and abundant unlabeled data. To address instability and overfitting arising from few-shot supervision under distribution drift, we introduce gradient-adaptive stabilization, a parameter-wise regularization mechanism implemented via gradient-scaled stochastic perturbations that promotes a principled stability-plasticity tradeoff. We further leverage unlabeled data through semi-supervised learning and introduce prototype anchored supervision that validates pseudo-labels via joint confidence and prototype consistency. Together, these mechanisms enable learning under joint nonstationarity in continual segmentation. Extensive empirical evaluation across class-incremental, domain-incremental, and few-shot regimes demonstrates consistent improvements over prior methods in heterogeneous structured prediction settings. Our results expose fundamental failure modes of existing continual segmentation approaches and provide insight into learning robust dense predictors in dynamically evolving environments.

Problem

• continual segmentation
• joint nonstationarity
• semantic classes
• distribution shift
• limited annotations

Innovation

• continual semantic segmentation
• joint nonstationarity
• gradient-adaptive stabilization
• prototype anchored supervision
• semi-supervised learning