Cracking the PUMA Challenge in 24 Hours with CellViT++ and nnU-Net

2025-03-15

AI Summary

This work addresses panoptic segmentation of melanoma histopathological images. We propose an efficient, deployable dual-model collaborative framework that decouples nuclear instance segmentation (CellViT++) from tissue region segmentation (nnU-Net), thereby circumventing the hyperparameter tuning complexity inherent in joint optimization. To our knowledge, this is the first end-to-end deployable paradigm achieving full pipeline deployment within 24 hours, incorporating a novel panoptic fusion post-processing step. Experimental results demonstrate a tissue segmentation Dice score of 0.750—representing a 12.1% improvement over the baseline—while nuclear detection achieves baseline-level performance on both F1-score and Aggregated Jaccard Index (AJI). The implementation is fully open-sourced, providing a lightweight, robust, and plug-and-play solution for clinical decision support in melanoma diagnosis.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Distributed Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Automatic tissue segmentation and nuclei detection is an important task in pathology, aiding in biomarker extraction and discovery. The panoptic segmentation of nuclei and tissue in advanced melanoma (PUMA) challenge aims to improve tissue segmentation and nuclei detection in melanoma histopathology. Unlike many challenge submissions focusing on extensive model tuning, our approach emphasizes delivering a deployable solution within a 24-hour development timeframe, using out-of-the-box frameworks. The pipeline combines two models, namely CellViT++ for nuclei detection and nnU-Net for tissue segmentation. Our results demonstrate a significant improvement in tissue segmentation, achieving a Dice score of 0.750, surpassing the baseline score of 0.629. For nuclei detection, we obtained results comparable to the baseline in both challenge tracks. The code is publicly available at https://github.com/TIO-IKIM/PUMA.

Problem

• Automated tissue segmentation and nuclei detection in melanoma histopathology.
• Improving panoptic segmentation for biomarker extraction and discovery.
• Developing a deployable solution within a 24-hour timeframe using existing frameworks.

Innovation

• Combines CellViT++ and nnU-Net models
• Focuses on deployable solution in 24 hours
• Achieves improved tissue segmentation Dice score