CytoFormer: A Molecularly Supervised Cell Foundation Model for Histopathology Cell Classification

2026-08-17

AI Summary

This study addresses the challenge of costly manual annotation in cell classification for H&E-stained pathology slides by proposing CytoFormer, a foundational model for cellular analysis. This approach innovatively leverages spatial transcriptomic molecular profiles as supervisory signals instead of manual labels, establishing a large-scale paired dataset spanning 16 organs alongside a multi-task learning framework. Experimental results demonstrate that CytoFormer achieves a cross-organ classification accuracy of 0.85, significantly outperforming existing methods in transferability. Furthermore, it attains an F1-score of 0.82 in few-shot scenarios, effectively enabling efficient and transferable cell-level analysis in routine histopathology without extensive annotation requirements.

Abstract

Identifying cell types directly from routine haematoxylin and eosin (H&E) histology would enable single-cell analysis at scale, but training such models has relied on manual pathologist annotations, which are slow, expensive and unreliable for many cell types. We instead supervise morphology with molecules. Imaging-based spatial transcriptomics profiles individual cells in situ on a section that can afterwards be stained with H&E, so that molecular identity and morphology are observed for the same physical cell. We assembled 81 such paired Xenium sections spanning 16 organs, derived per-cell labels by clustering, marker-gene annotation, organ-wise human review and quality control, and mapped them onto the cell types commonly reported in each organ. This yielded 15.4 million cells, each with a paired H&E image patch and one of 23 cell types, on which we trained CytoFormer, a cell foundation model with a multi-task, per-organ classification head. On spatially held-out tissue CytoFormer reached an accuracy of 0.85 and a macro-F1 of 0.78 across all 16 organs, and its predictions reproduced the tissue architecture of an entire held-out section. The representation also transfers: with the encoder frozen, a linear head on CytoFormer features performed better than six pathology foundation models on four expert-annotated benchmarks, including on organs and cell types that were not part of pretraining. Finally, in an interactive active-learning setting, CytoFormer's embeddings are markedly more label-efficient than existing pathology foundation models, detecting normal epithelium amid look-alike tumour with an F1 of 0.82 from only a few annotations and leading the strongest baseline by 0.13 in F1. CytoFormer turns paired H&E and spatial transcriptomics into a reusable, label-efficient representation for cell-level analysis of routine histology.

Problem

• Cell Classification
• Histopathology
• H&E Staining
• Manual Annotation
• Spatial Transcriptomics

Innovation

• Molecular Supervision
• Cell Foundation Model
• Spatial Transcriptomics
• Label Efficiency
• Histopathology