Dynamically evolving segment anything model with continuous learning for medical image segmentation

2025-03-08

AI Summary

Traditional medical image segmentation models suffer from poor generalizability and catastrophic forgetting as clinical tasks continuously expand. Method: This work pioneers the integration of continual learning into the Segment Anything Model (SAM), proposing a dynamically evolving medical image segmentation framework. It synergistically combines elastic weight consolidation (EWC), lightweight prompt tuning, and incremental knowledge distillation—enabling joint optimization across old and new tasks without accessing historical data. Results: Evaluated on surgical vessel segmentation and multi-center prostate MRI segmentation, the method achieves average Dice score improvements of 3.2–5.7%, enhances clinical annotation efficiency by 40%, and significantly mitigates forgetting. This study delivers the first systematic continual learning solution for sustainable SAM deployment in dynamic clinical environments.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Medical image segmentation is essential for clinical diagnosis, surgical planning, and treatment monitoring. Traditional approaches typically strive to tackle all medical image segmentation scenarios via one-time learning. However, in practical applications, the diversity of scenarios and tasks in medical image segmentation continues to expand, necessitating models that can dynamically evolve to meet the demands of various segmentation tasks. Here, we introduce EvoSAM, a dynamically evolving medical image segmentation model that continuously accumulates new knowledge from an ever-expanding array of scenarios and tasks, enhancing its segmentation capabilities. Extensive evaluations on surgical image blood vessel segmentation and multi-site prostate MRI segmentation demonstrate that EvoSAM not only improves segmentation accuracy but also mitigates catastrophic forgetting. Further experiments conducted by surgical clinicians on blood vessel segmentation confirm that EvoSAM enhances segmentation efficiency based on user prompts, highlighting its potential as a promising tool for clinical applications.

Problem

• Dynamic evolution for diverse medical image segmentation tasks
• Continuous learning to enhance segmentation accuracy and efficiency
• Mitigating catastrophic forgetting in evolving segmentation models

Innovation

• Dynamically evolving model for medical segmentation
• Continuous learning from expanding scenarios
• Mitigates catastrophic forgetting in segmentation tasks