SelfAdapt: Unsupervised Domain Adaptation of Cell Segmentation Models

2025-08-15

AI Summary

To address the substantial performance degradation of pretrained models (e.g., Cellpose) in cross-domain biomedical instance segmentation, this paper proposes an unsupervised domain adaptation (UDA) method requiring no target-domain annotations. The approach adopts a student–teacher framework integrating augmented consistency training, L2-SP weight regularization, and an unlabeled stopping criterion to robustly adapt source-pretrained models. It is the first work to introduce UDA to cell-level instance segmentation and further enhances models already fine-tuned under supervision. Evaluated on LiveCell and TissueNet, our method achieves up to a 29.64% relative improvement in AP₀.₅ over the original Cellpose—significantly outperforming existing unsupervised baselines. These results demonstrate both effectiveness and strong generalizability across diverse biological imaging domains.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Medical and Biological Imaging
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Deep neural networks have become the go-to method for biomedical instance segmentation. Generalist models like Cellpose demonstrate state-of-the-art performance across diverse cellular data, though their effectiveness often degrades on domains that differ from their training data. While supervised fine-tuning can address this limitation, it requires annotated data that may not be readily available. We propose SelfAdapt, a method that enables the adaptation of pre-trained cell segmentation models without the need for labels. Our approach builds upon student-teacher augmentation consistency training, introducing L2-SP regularization and label-free stopping criteria. We evaluate our method on the LiveCell and TissueNet datasets, demonstrating relative improvements in AP0.5 of up to 29.64% over baseline Cellpose. Additionally, we show that our unsupervised adaptation can further improve models that were previously fine-tuned with supervision. We release SelfAdapt as an easy-to-use extension of the Cellpose framework. The code for our method is publicly available at https: //github.com/Kainmueller-Lab/self_adapt.

Problem

• Adapts cell segmentation models without labeled data
• Improves performance on domains differing from training data
• Enhances models pre-tuned with supervised methods

Innovation

• Unsupervised domain adaptation for cell segmentation
• Student-teacher augmentation consistency training
• L2-SP regularization and label-free stopping criteria