BiTro: Bidirectional Transfer Learning Enhances Bulk and Spatial Transcriptomics Prediction in Cancer Pathological Images

2026-03-16

AI Summary

This study addresses the modality gap between histopathological images and transcriptomic data in cancer research: bulk RNA-seq lacks spatial resolution, while spatial transcriptomics suffers from high cost and limited sample availability, hindering precise cross-modal mapping. To bridge this gap, the authors propose BiTro, a bidirectional transfer learning framework that leverages unified cell-level histopathology modeling and multiple instance learning to map morphological and spatial features onto both bulk and spatial transcriptomes. BiTro introduces, for the first time, a LoRA-based bidirectional transfer mechanism to efficiently integrate the complementary information from both transcriptomic modalities. Experiments across five cancer datasets demonstrate that BiTro achieves state-of-the-art or competitive performance in predicting both bulk and spatial transcriptomic profiles, with transfer learning significantly enhancing predictive accuracy.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Cancer pathological analysis requires modeling tumor heterogeneity across multiple modalities, primarily through transcriptomics and whole slide imaging (WSI), along with their spatial relations. On one hand, bulk transcriptomics and WSI images are largely available but lack spatial mapping; on the other hand, spatial transcriptomics (ST) data can offer high spatial resolution, yet facing challenges of high cost, low sequencing depth, and limited sample sizes. Therefore, the data foundation of either side is flawed and has its limit in accurately finding the mapping between the two modalities. To this end, we propose BiTro, a bidirectional transfer learning framework that can enhance bulk and spatial transcriptomics prediction from pathological images. Our contributions are twofold. First, we design a universal and transferable model architecture that works for both bulk+WSI and ST data. A major highlight is that we model WSI images on the cellular level to better capture cells' visual features, morphological phenotypes, and their spatial relations; to map cells' features to their transcriptomics measured in bulk or ST, we adopt multiple instance learning. Second, by using LoRA, our model can be efficiently transferred between bulk and ST data to exploit their complementary information. To test our framework, we conducted comprehensive experiments on five cancer datasets. Results demonstrate that 1) our base model can achieve better or competitive performance compared to existing models on bulk or spatial transcriptomics prediction, and 2) transfer learning can further improve the base model's performance.

Problem

• bulk transcriptomics
• spatial transcriptomics
• whole slide imaging
• tumor heterogeneity
• multimodal mapping

Innovation

• Bidirectional Transfer Learning
• Spatial Transcriptomics
• Multiple Instance Learning
• LoRA
• Cell-level Modeling