Continuous Online Adaptation Driven by User Interaction for Medical Image Segmentation

2025-03-09

AI Summary

To address performance degradation of medical image segmentation models in deployment due to distribution shift, this paper proposes a clinical-interaction-oriented online continual learning framework. Methodologically, we introduce a Gaussian point loss to encode physician click priors and design a two-stage lightweight online optimization mechanism—enabling, for the first time, unsupervised, continual, and low-overhead model self-updating at test time. The framework requires neither annotated data nor full retraining, and supports zero-shot cross-modality (e.g., fundus → brain MRI) and cross-lesion generalization. Extensive experiments on five fundus and four brain MRI datasets demonstrate that our approach significantly outperforms existing interactive segmentation and online learning methods, markedly enhancing model robustness and adaptability in dynamic clinical environments.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Interactive segmentation models use real-time user interactions, such as mouse clicks, as extra inputs to dynamically refine the model predictions. After model deployment, user corrections of model predictions could be used to adapt the model to the post-deployment data distribution, countering distribution-shift and enhancing reliability. Motivated by this, we introduce an online adaptation framework that enables an interactive segmentation model to continuously learn from user interaction and improve its performance on new data distributions, as it processes a sequence of test images. We introduce the Gaussian Point Loss function to train the model how to leverage user clicks, along with a two-stage online optimization method that adapts the model using the corrected predictions generated via user interactions. We demonstrate that this simple and therefore practical approach is very effective. Experiments on 5 fundus and 4 brain MRI databases demonstrate that our method outperforms existing approaches under various data distribution shifts, including segmentation of image modalities and pathologies not seen during training.

Problem

• Enhance medical image segmentation via user interaction.
• Adapt models to new data distributions post-deployment.
• Improve segmentation accuracy across diverse medical imaging modalities.

Innovation

• Online adaptation framework for interactive segmentation
• Gaussian Point Loss leverages user clicks
• Two-stage online optimization enhances model performance