Evaluating New AI Cell Foundation Models on Challenging Kidney Pathology Cases Unaddressed by Previous Foundation Models

2025-09-30

AI Summary

Nuclear segmentation in renal histopathology remains challenging due to high tissue morphological heterogeneity and imaging variability, leading to suboptimal accuracy. Method: We systematically evaluate state-of-the-art AI foundation models—including CellViT++ (Virchow) and Cellpose-SAM—on previously unaddressed complex renal cases, introducing a fusion-based ensemble evaluation framework that integrates multi-model consensus analysis with expert pathologist scoring to ensure robust segmentation of challenging samples. Contribution/Results: Experiments demonstrate that the fusion ensemble significantly improves performance: the “good” segmentation rate rises from 40.3% (CellViT++ alone) to 62.2%, while the “poor” rating drops sharply to 0.4%. This work pioneers the integration of foundation model ensembling with a clinically driven, difficult-case–oriented evaluation paradigm, establishing a novel methodological framework and empirical benchmark for trustworthy AI deployment in renal digital pathology.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Accurate cell nuclei segmentation is critical for downstream tasks in kidney pathology and remains a major challenge due to the morphological diversity and imaging variability of renal tissues. While our prior work has evaluated early-generation AI cell foundation models in this domain, the effectiveness of recent cell foundation models remains unclear. In this study, we benchmark advanced AI cell foundation models (2025), including CellViT++ variants and Cellpose-SAM, against three widely used cell foundation models developed prior to 2024, using a diverse large-scale set of kidney image patches within a human-in-the-loop rating framework. We further performed fusion-based ensemble evaluation and model agreement analysis to assess the segmentation capabilities of the different models. Our results show that CellViT++ [Virchow] yields the highest standalone performance with 40.3% of predictions rated as "Good" on a curated set of 2,091 challenging samples, outperforming all prior models. In addition, our fused model achieves 62.2% "Good" predictions and only 0.4% "Bad", substantially reducing segmentation errors. Notably, the fusion model (2025) successfully resolved the majority of challenging cases that remained unaddressed in our previous study. These findings demonstrate the potential of AI cell foundation model development in renal pathology and provide a curated dataset of challenging samples to support future kidney-specific model refinement.

Problem

• Evaluating new AI cell foundation models on challenging kidney pathology cases
• Benchmarking advanced models against prior versions for nuclei segmentation
• Assessing segmentation capabilities using human-in-the-loop rating framework

Innovation

• Benchmarking advanced CellViT++ and Cellpose-SAM models
• Implementing fusion-based ensemble evaluation for segmentation
• Using human-in-the-loop rating framework for validation