Trimodal Glioma Representation Alignment via Volumetric Contrastive Learning

2026-06-12

AI Summary

Accurate glioma grading and survival prediction require the integration of histopathological, genomic, and MRI data; however, existing approaches are largely limited to bimodal fusion and suffer from inadequate cross-modal alignment. This work proposes GLORIA, a novel framework that employs modality-specific encoders to extract features from whole-slide images, gene expression profiles, and 3D MRI scans, projecting them into a shared latent space. To achieve effective trilinear alignment, we introduce a Gram contrastive loss based on the volume spanned by triplet embeddings—a first in multimodal learning. A cross-modal gating module subsequently fuses these aligned representations, enabling joint optimization for both tumor grade classification (three-tier) and overall survival prediction. Evaluated on a matched cohort of 132 cases from TCGA-GBM/LGG and BraTS21, GLORIA significantly outperforms bimodal baselines across all metrics, establishing the first end-to-end model that cohesively integrates histopathology, genomics, and MRI.

Abstract

Glioma grading and survival prediction require the integration of heterogeneous information collected at different spatial and biological scales. Histopathology describes tissue morphology, mRNA expression captures molecular activity, and magnetic resonance imaging provides a non-invasive view of tumor extent and radiological heterogeneity. Existing glioma prognosis models often combine only two of these sources, while their alignment objectives remain mostly pairwise. This paper introduces GLORIA, a novel trimodal framework for GLioma Omics - Radiology - hIstopathology Alignment. GLORIA processes whole-slide image regions, gene-expression profiles, and 3D MRI volumes through modality-specific encoders, projects them into a shared latent space, and aligns them with a Gramian contrastive loss that measures the volume spanned by the three modality embeddings. The aligned representations are fused through a cross-modal gating module and optimized jointly for three-class glioma grading and overall survival prediction. We evaluate GLORIA on a matched TCGA-GBM/LGG and BraTS21 cohort, comprising 132 patients with all three modalities. On the shared trimodal test set, GLORIA improves over the bimodal WSI-mRNA baseline in all the metrics considered.

Problem

• glioma
• trimodal alignment
• heterogeneous data integration
• survival prediction
• multimodal representation

Innovation

• trimodal alignment
• volumetric contrastive learning
• Gramian contrastive loss
• cross-modal gating
• glioma prognosis