Genome-Anchored Foundation Model Embeddings Improve Molecular Prediction from Histology Images

2025-06-24

AI Summary

A key challenge in precision oncology is predicting complex molecular features and patient prognosis directly from routine whole-slide images (WSIs), bypassing costly and time-consuming genomic assays. To address this, we propose PathLUPI—the first method to leverage transcriptomic data as “privileged information” (LUPI paradigm) during training, thereby constructing a genome-anchored histopathological embedding space. Crucially, PathLUPI enables accurate WSI-driven inference of molecular phenotypes and survival risk without requiring molecular data at test time. Built upon a multiple-instance learning framework, it integrates self-supervised contrastive learning to refine histologic representations. Evaluated across 20 cohorts comprising 11,257 samples and 49 prediction tasks, PathLUPI achieves AUC ≥ 0.80 on 14 biomarker prediction tasks and C-index ≥ 0.70 on survival prediction for five cancer types. This advances interpretable, clinically deployable computational pathology.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Search and Optimization: Learning to Search
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Precision oncology requires accurate molecular insights, yet obtaining these directly from genomics is costly and time-consuming for broad clinical use. Predicting complex molecular features and patient prognosis directly from routine whole-slide images (WSI) remains a major challenge for current deep learning methods. Here we introduce PathLUPI, which uses transcriptomic privileged information during training to extract genome-anchored histological embeddings, enabling effective molecular prediction using only WSIs at inference. Through extensive evaluation across 49 molecular oncology tasks using 11,257 cases among 20 cohorts, PathLUPI demonstrated superior performance compared to conventional methods trained solely on WSIs. Crucially, it achieves AUC $geq$ 0.80 in 14 of the biomarker prediction and molecular subtyping tasks and C-index $geq$ 0.70 in survival cohorts of 5 major cancer types. Moreover, PathLUPI embeddings reveal distinct cellular morphological signatures associated with specific genotypes and related biological pathways within WSIs. By effectively encoding molecular context to refine WSI representations, PathLUPI overcomes a key limitation of existing models and offers a novel strategy to bridge molecular insights with routine pathology workflows for wider clinical application.

Problem

• Predict molecular features from histology images efficiently
• Improve accuracy in oncology tasks using genome-anchored embeddings
• Bridge molecular insights with routine pathology workflows

Innovation

• Uses transcriptomic privileged information for training
• Generates genome-anchored histological embeddings from WSIs
• Improves molecular prediction accuracy in oncology tasks