Transcriptomic Models for Immunotherapy Response Prediction Show Limited Cross-cohort Generalisability

2026-04-07

AI Summary

Current transcriptomic models for predicting immune checkpoint inhibitor (ICI) response commonly suffer from poor cross-cohort generalizability and unclear biological interpretability. This study presents the first systematic evaluation of nine state-of-the-art models—five based on bulk RNA-seq and four on single-cell RNA-seq—including representative methods such as COMPASS, IRNet, and NetBio, assessing both predictive performance and biological signal consistency across independent cohorts. Through large-scale benchmarking and pathway enrichment analyses, we find that most models perform near random levels, with single-cell models offering only marginal improvements, and that the biomarkers identified by different models exhibit substantial inconsistency. Our work highlights critical limitations in domain adaptability and interpretability among existing ICI response prediction approaches and establishes a foundational benchmark to guide future methodological development.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Immune checkpoint inhibitors (ICIs) have transformed cancer therapy; yet substantial proportion of patients exhibit intrinsic or acquired resistance, making accurate pre-treatment response prediction a critical unmet need. Transcriptomics-based biomarkers derived from bulk and single-cell RNA sequencing (scRNA-seq) offer a promising avenue for capturing tumour-immune interactions, yet the cross-cohort generalisability of existing prediction models remains unclear.We systematically benchmark nine state-of-the-art transcriptomic ICI response predictors, five bulk RNA-seq-based models (COMPASS, IRNet, NetBio, IKCScore, and TNBC-ICI) and four scRNA-seq-based models (PRECISE, DeepGeneX, Tres and scCURE), using publicly available independent datasets unseen during model development. Overall, predictive performance was modest: bulk RNA-seq models performed at or near chance level across most cohorts, while scRNA-seq models showed only marginal improvements. Pathway-level analyses revealed sparse and inconsistent biomarker signals across models. Although scRNA-seq-based predictors converged on immune-related programs such as allograft rejection, bulk RNA-seq-based models exhibited little reproducible overlap. PRECISE and NetBio identified the most coherent immune-related themes, whereas IRNet predominantly captured metabolic pathways weakly aligned with ICI biology. Together, these findings demonstrate the limited cross-cohort robustness and biological consistency of current transcriptomic ICI prediction models, underscoring the need for improved domain adaptation, standardised preprocessing, and biologically grounded model design.

Problem

• immunotherapy response prediction
• transcriptomic models
• cross-cohort generalisability
• immune checkpoint inhibitors
• biomarker reproducibility

Innovation

• cross-cohort generalisability
• transcriptomic biomarkers
• immune checkpoint inhibitors
• bulk RNA-seq
• scRNA-seq