Rethinking the Adaptation of Vision Foundation Models for Efficient Cell Segmentation

2026-06-20

AI Summary

This work addresses the challenge of efficiently adapting large vision foundation models to cell segmentation, which typically requires computationally expensive fine-tuning and abundant annotated data. The authors propose EffiCell-Seg, a novel framework that operates with a frozen pretrained vision encoder and introduces, for the first time, an insight into its inherent complementary priors—global saliency and local morphology. Leveraging this observation, they design a training-free adaptation mechanism: a Cell Structure Prompt Encoder generates structural prior maps, which are synergistically integrated with a Mask Decoder that mutually guides predictions using geometric distance fields and semantic maps. Evaluated across diverse cellular imaging modalities, EffiCell-Seg achieves state-of-the-art performance while employing only approximately 5 million trainable parameters—over 130 times fewer than full fine-tuning.

Abstract

Cell segmentation is critical for computational pathology and biomedical discovery. While recent Vision Foundation Models (VFMs) have demonstrated remarkable universal feature representations, unlocking their full potential for cellular imaging is currently bottlenecked by resource-intensive adaptation paradigms. Existing methods typically rely on fine-tuning heavy visual encoders, leading to extensive computational overhead and a dependency on large-scale annotations. To address this, we propose the EffiCell-Seg framework for highly efficient cell segmentation without re-training the visual encoder. Our core insight is that pretrained VFMs intrinsically encode complementary structural priors: global saliency for localizing potential cells, and local morphological patterns for delineating cellular structures. To harness these priors, we devise a Cell Structure Prompt Encoder (CSP-Encoder) that synthesizes semantic-aware saliency and principal morphological features from frozen VFM representations into explicit structural prior maps. Moreover, we propose a Synergistic Mask Decoder (SM-Decoder) that enforces contextual consistency by jointly predicting geometric distance fields and semantic maps via mutual cross-guidance. Extensive experiments demonstrate that EffiCell-Seg outperforms state-of-the-art methods across diverse cell imaging modalities while requiring only ~5M trainable parameters, over 130x fewer than fully fine-tuned VFM counterparts. The code is available at https://github.com/xq141839/EffiCell-Seg.

Problem

• cell segmentation
• Vision Foundation Models
• efficient adaptation
• computational pathology
• resource-intensive

Innovation

• Vision Foundation Models
• Cell Segmentation
• Parameter-Efficient Adaptation
• Structural Priors
• Prompt Encoding