π€ AI Summary
This work addresses the challenge of batch effects across institutions in computational pathology, which often distort biological features and impair cross-institutional generalization of foundation models. To mitigate this, the study introduces a language-mediated embedding generation framework (GLMP) that leverages a general-purpose multimodal large language model (MLLM). GLMP first translates histopathology images into textual descriptions and then encodes these texts into numerical embeddings using a text encoder, thereby enhancing biologically relevant signals while suppressing institution-specific artifacts. This approach marks the first use of textual representations as an intermediate modality in computational pathology. Experimental results demonstrate that GLMP substantially improves cross-institutional generalization performance, validating the efficacy and potential of general-purpose large models in pathological image analysis.
π Abstract
Pathology foundation models (PFMs) have demonstrated strong potential across clinical and scientific applications, yet their performance is often hindered by batch effects, which are non-biological variations across tissue source institutions (TSIs) that distort learned feature representations and impair generalization. Conventional mitigation strategies, such as stain normalization, offer limited success in addressing these high-dimensional, complex artifacts. We present GLMP (General-purpose LLM-Mediated Pathology model), a novel framework that generates robust numerical embeddings from histology image patches through an intermediate textual representation. By leveraging pretrained general-purpose multimodal large language models (MLLMs) and text encoders, GLMP effectively prioritizes biologically meaningful signals over TSI-specific artifacts, thereby improving cross-institutional generalization. To our knowledge, GLMP is the first pathology model to use text descriptions of histological features as an intermediate representation for generating numerical embeddings from histology images. Our results highlight the untapped potential of broad-domain, non-specialized MLLMs in computational pathology and introduce a new paradigm for building versatile, generalizable, and robust pathology models.