Sketchpose: Learning to Segment Cells with Partial Annotations

2025-08-25

AI Summary

Existing cell segmentation models (e.g., Cellpose, StarDist, HoverNet) rely on fully annotated distance maps—costly to produce and limiting transfer learning and low-resource deployment. This work introduces the first distance-map regression framework supporting *partial annotations*. By incorporating an unlabeled-region mask and a robust loss function, it enables accurate distance-map prediction from sparse boundary or point annotations—without architectural modifications and compatible with mainstream backbones. Integrated into a Napari plugin, the method supports interactive annotation and inference. Evaluated on multiple public benchmarks, it matches or exceeds fully supervised baselines in segmentation accuracy (average Dice score improved by 0.3–1.2 percentage points), while reducing annotation effort by 60–85%. This advances frugal learning and cross-domain adaptation in biomedical image analysis.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

The most popular networks used for cell segmentation (e.g. Cellpose, Stardist, HoverNet,...) rely on a prediction of a distance map. It yields unprecedented accuracy but hinges on fully annotated datasets. This is a serious limitation to generate training sets and perform transfer learning. In this paper, we propose a method that still relies on the distance map and handles partially annotated objects. We evaluate the performance of the proposed approach in the contexts of frugal learning, transfer learning and regular learning on regular databases. Our experiments show that it can lead to substantial savings in time and resources without sacrificing segmentation quality. The proposed algorithm is embedded in a user-friendly Napari plugin.

Problem

• Handles partially annotated objects for cell segmentation
• Reduces reliance on fully annotated training datasets
• Enables frugal and transfer learning without quality loss

Innovation

• Handles partially annotated objects
• Relies on distance map prediction
• Embedded in user-friendly Napari plugin