Tissue Classification and Whole-Slide Images Analysis via Modeling of the Tumor Microenvironment and Biological Pathways

2026-01-13

AI Summary

This study addresses the limitation of existing methods that primarily focus on single-gene sequences or patch-level classification, which struggle to effectively integrate spatial transcriptomics with histological image patches for comprehensive characterization of the tumor microenvironment. To overcome this, the authors propose BioMorphNet, a multimodal network that leverages graph neural networks to model spatial relationships among image patches and fuses morphological features with spatially resolved gene expression. A novel learnable pathway module is introduced, which incorporates prior knowledge from predefined pathway databases to bridge morphological and molecular data, enabling synergistic representation learning. Evaluated on prostate, colorectal, and breast cancer datasets, the method achieves accuracy improvements of 2.67%, 5.48%, and 6.29%, respectively, while also facilitating precise tumor localization and the discovery of potential biomarkers.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Automatic integration of whole slide images (WSIs) and gene expression profiles has demonstrated substantial potential in precision clinical diagnosis and cancer progression studies. However, most existing studies focus on individual gene sequences and slide level classification tasks, with limited attention to spatial transcriptomics and patch level applications. To address this limitation, we propose a multimodal network, BioMorphNet, which automatically integrates tissue morphological features and spatial gene expression to support tissue classification and differential gene analysis. For considering morphological features, BioMorphNet constructs a graph to model the relationships between target patches and their neighbors, and adjusts the response strength based on morphological and molecular level similarity, to better characterize the tumor microenvironment. In terms of multimodal interactions, BioMorphNet derives clinical pathway features from spatial transcriptomic data based on a predefined pathway database, serving as a bridge between tissue morphology and gene expression. In addition, a novel learnable pathway module is designed to automatically simulate the biological pathway formation process, providing a complementary representation to existing clinical pathways. Compared with the latest morphology gene multimodal methods, BioMorphNet's average classification metrics improve by 2.67%, 5.48%, and 6.29% for prostate cancer, colorectal cancer, and breast cancer datasets, respectively. BioMorphNet not only classifies tissue categories within WSIs accurately to support tumor localization, but also analyzes differential gene expression between tissue categories based on prediction confidence, contributing to the discovery of potential tumor biomarkers.

Problem

• whole-slide image analysis
• spatial transcriptomics
• tumor microenvironment
• tissue classification
• multimodal integration

Innovation

• multimodal integration
• spatial transcriptomics
• tumor microenvironment
• learnable pathway module
• graph-based morphology modeling