What to Preserve, Where to Adapt: A Depth-Wise Analysis of Forgetting in Continual Gynecological Image Segmentation

2026-08-13

AI Summary

This study addresses catastrophic forgetting induced by data distribution shifts in continual gynecological image segmentation by systematically analyzing the spatial characteristics of forgetting along the network depth dimension. Through block-wise ablation, controlled parameter adaptation, and hierarchical evaluation experiments, this work elucidates how updates across different encoder-decoder regions influence forgetting mechanisms. The findings reveal that early encoder and late decoder layers are most sensitive to performance degradation, whereas updating bottleneck-adjacent regions significantly mitigates forgetting. By identifying critical network zones essential for alleviating catastrophic forgetting, this research provides a vital theoretical foundation and methodological guidance for advancing continual learning in medical image segmentation.

Abstract

Medical image segmentation models are typically trained under the assumption that all data are available simultaneously. However, in clinical practice, datasets often arrive sequentially, requiring models to adapt continuously to evolving data distributions. We study this problem in gynecological image segmentation, where substantial heterogeneity across imaging modalities, anatomical structures, and annotation protocols creates a particularly challenging continual learning setting. Under these large distribution shifts, existing continual learning methods struggle to preserve previously learned knowledge, leading to catastrophic forgetting. To better understand forgetting in this setting, we investigate how different encoder--decoder regions influence segmentation performance and forgetting during continual gynecological segmentation. Through block-wise ablation analysis, we observe that ablating early encoder and late decoder regions results in the largest performance degradation, indicating that segmentation performance depends unevenly across the network hierarchy. Using controlled adaptation experiments, we further show that forgetting remains limited when updates are restricted to bottleneck-adjacent regions, but increases sharply once shallower encoders and decoders become trainable, even when only a small subset of parameters is updated. These findings suggest that forgetting in the encoder-decoder architecture is strongly influenced by where updates occur across network depth during continual learning. Full code and analysis pipelines will be made publicly available upon acceptance.

Problem

• Continual Learning
• Catastrophic Forgetting
• Gynecological Image Segmentation
• Distribution Shift

Innovation

• Continual Learning
• Catastrophic Forgetting
• Depth-wise Analysis
• Gynecological Image Segmentation
• Encoder-Decoder Architecture