A Foundation Model for Spatial Proteomics

📅 2025-06-03
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
Spatial proteomics lacks foundational models tailored for high-dimensional, multi-channel, heterogeneous fluorescence imaging data. This paper introduces KRONOS—the first foundation model designed specifically for single-cell-resolution spatial protein images. KRONOS innovatively adopts a segmentation-free, patch-level self-supervised learning paradigm and incorporates a dedicated multi-scale encoder architecture to enable cross-platform normalization and biologically interpretable representation learning—spanning cellular, microenvironmental, and tissue-level contexts. Leveraging patch embedding retrieval and lightweight downstream head adaptation, KRONOS achieves state-of-the-art performance across 11 independent cohorts on cell phenotyping classification, therapy response prediction, and spatial pattern retrieval. It significantly enhances few-shot generalization capability and supports cross-institutional standardized analysis and reverse search of spatial images.

Technology Category

Application Category

📝 Abstract
Foundation models have begun to transform image analysis by acting as pretrained generalist backbones that can be adapted to many tasks even when post-training data are limited, yet their impact on spatial proteomics, imaging that maps proteins at single-cell resolution, remains limited. Here, we introduce KRONOS, a foundation model built for spatial proteomics. KRONOS was trained in a self-supervised manner on over 47 million image patches covering 175 protein markers, 16 tissue types, and 8 fluorescence-based imaging platforms. We introduce key architectural adaptations to address the high-dimensional, multi-channel, and heterogeneous nature of multiplex imaging. We demonstrate that KRONOS learns biologically meaningful representations across multiple scales, ranging from cellular and microenvironment to tissue levels, enabling it to address diverse downstream tasks, including cell phenotyping, region classification, and patient stratification. Evaluated across 11 independent cohorts, KRONOS achieves state-of-the-art performance across cell phenotyping, treatment response prediction, and retrieval tasks, and is highly data-efficient. KRONOS also introduces the paradigm of segmentation-free patch-level processing for efficient and scalable spatial proteomics analysis, allowing cross-institutional comparisons, and as an image reverse search engine for spatial patterns. Together, these results position KRONOS as a flexible and scalable tool for spatial proteomics. The model is publicly accessible at https://github.com/mahmoodlab/KRONOS.
Problem

Research questions and friction points this paper is trying to address.

Developing a foundation model for spatial proteomics analysis
Addressing high-dimensional multi-channel multiplex imaging challenges
Enabling segmentation-free scalable protein pattern analysis
Innovation

Methods, ideas, or system contributions that make the work stand out.

Self-supervised training on 47M protein image patches
Adapts architecture for multi-channel multiplex imaging
Segmentation-free patch-level processing for scalability
🔎 Similar Papers
No similar papers found.
Authors to Follow
Muhammad Shaban
Muhammad Shaban
Brigham and Women's Hospital, Harvard Medical School
Computational PathologyDeep Learning
Yuzhou Chang
Yuzhou Chang
Joint Post Doc researcher (OSU/BIDMC)
graph signal processingspatial omicsscRNA-seqbioinformatics
H
Huaying Qiu
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
Yao Yu Yeo
Yao Yu Yeo
Harvard University
Spatial OmicsSystems ImmunologyVirologyCancerTechnology Development
Andrew H. Song
Andrew H. Song
Postdoctoral fellow, Harvard Medical School
Computational pathologyStatistical signal processing
Guillaume Jaume
Guillaume Jaume
Harvard Medical School, Brigham and Women's Hospital
Computational Pathology - Computer Vision
Y
Yuchen Wang
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
L
Luca L. Weishaupt
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Broad Institute of Harvard and MIT, Cambridge, MA, USA; Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
Tong Ding
Tong Ding
PhD student in Computer Science, Harvard University
Representation LearningComputer VisionMultimodal LearningMachine Learning for Health
Anurag Vaidya
Anurag Vaidya
MIT
Multimodal frontier modelsagentsclinical deployment
A
Abdallah Lamane
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Broad Institute of Harvard and MIT, Cambridge, MA, USA
Daniel Shao
Daniel Shao
PhD Candidate, Massachusetts Institute of Technology
machine learningfoundation modelscomputer vision
M
Mohammed Zidane
Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
Y
Yunhao Bai
Broad Institute of Harvard and MIT, Cambridge, MA, USA
P
Paige McCallum
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA; Division of Experimental Medicine, McGill University; Montreal, Quebec, Canada
S
Shuli Luo
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
W
Wenrui Wu
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
Y
Yang Wang
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
P
Precious Cramer
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
C
Chi Ngai Chan
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
P
Pierre Stephan
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
J
Johanna Schaffenrath
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
J
Jia Le Lee
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
H
Hendrik A. Michel
Center for Virology and Vaccine Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
C
Caiwei Tian
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Broad Institute of Harvard and MIT, Cambridge, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
C
Cristina Almagro-Perez
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Broad Institute of Harvard and MIT, Cambridge, MA, USA
Sophia J. Wagner
Sophia J. Wagner
Technical University Munich, Helmholtz AI
computational pathologydeep learningcomputer vision
Sharifa Sahai
Sharifa Sahai
PhD Candidate, Harvard University
computational pathologydeep learningmedical imaging
Ming Y. Lu
Ming Y. Lu
MIT EECS, Harvard Medical School
Computational PathologyComputer VisionNatural Language Processing
Richard J. Chen
Richard J. Chen
Brigham and Women's Hospital, Harvard Medical School
Representation LearningComputer VisionComputational Pathology
Andrew Zhang
Andrew Zhang
PhD student, Harvard & MIT
computer visionartificial intelligencehealthcaremedical devicesneuroscience
Mark Edward M. Gonzales
Mark Edward M. Gonzales
A*STAR, Singapore
BioinformaticsComputer visionNatural language processingMachine learningData science
A
Ahmad Makky
Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
J
Jia-Ying Joey Lee
Bioinformatics Institute (BII), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
H
Hao Cheng
Pelotonia Institute for Immuno-Oncology, The James Comprehensive Cancer Center, The Ohio State University, Columbus, OH, USA
N
Nourhan El Ahmar
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
S
Sayed Matar
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
M
Maximilian Haist
Department of Microbiology and Immunology, Stanford University, Stanford, CA, USA; Department of Pathology, Stanford University, Stanford, CA, USA
D
Darci Phillips
Department of Microbiology and Immunology, Stanford University, Stanford, CA, USA; Department of Pathology, Stanford University, Stanford, CA, USA
Y
Yuqi Tan
Department of Microbiology and Immunology, Stanford University, Stanford, CA, USA; Department of Pathology, Stanford University, Stanford, CA, USA
G
Garry P. Nolan
Department of Microbiology and Immunology, Stanford University, Stanford, CA, USA; Department of Pathology, Stanford University, Stanford, CA, USA
W
W. Richard Burack
Department of Pathology and Laboratory Medicine, University of Rochester Medical Center, Rochester, NY, USA
J
Jacob D. Estes
Vaccine & Gene Therapy Institute, Oregon Health & Science University, Beaverton, OR, USA
Jonathan T.C. Liu
Jonathan T.C. Liu
Professor, Stanford University
Biomedical opticsNondestructive 3D pathologyComputational pathologyAI analysis
T
Toni K. Choueiri
Lank Center for Genitourinary (GU) Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
N
Neeraj Agarwal
Department of Pathology, University of Utah, Salt Lake City, UT; ARUP Institute for Clinical and Experimental Pathology, Salt Lake City, UT, USA
M
Marc Barry
Department of Pathology, University of Utah, Salt Lake City, UT; ARUP Institute for Clinical and Experimental Pathology, Salt Lake City, UT, USA
S
Scott J. Rodig
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
G
Georg Gerber
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
C
Christian M. Schürch
Cluster of Excellence iFIT (EXC 2180) “Image-Guided and Functionally Instructed Tumor Therapies”, University of Tübingen, Germany
F
Fabian J. Theis
Institute of Computational Biology, Helmholtz Center, Munich, Germany
Y
Youn H Kim
Department of Dermatology, Stanford University, Stanford, CA, USA
J
Joe Yeong
Institute of Molecular and Cell Biology (IMCB), Agency for Science Technology and Research (A*STAR), Singapore, Singapore
S
Sabina Signoretti
Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
B
Brooke E. Howitt
Department of Pathology, Stanford University, Stanford, CA, USA
Lit-Hsin Loo
Lit-Hsin Loo
Bioinformatics Institute, A*STAR
Spatial OmicsHigh-throughput ImagingNew Approach MethodsComputational Pharmacology
Qin Ma
Qin Ma
Professor and Vice Chair for Research, Dept. of Biomedical Informatics, The Ohio State University
Single-cell multi-omicsGraph representation learningBiomedical InformaticsImmuno-informatics
Sizun Jiang
Sizun Jiang
Beth Israel Deaconess Medical Center/Harvard Medical School
Multiplexed ImagingSystems ImmunologyVirologyCancerTechnology Development
Faisal Mahmood
Faisal Mahmood
Associate Professor, Harvard University