Counterfactual Understanding via Retrieval-aware Multimodal Modeling for Time-to-Event Survival Prediction

2026-02-23

AI Summary

This work addresses the challenge of individualized counterfactual time-to-event survival prediction under heterogeneity and censoring by proposing the CURE framework. CURE integrates multimodal data—including clinical, paraclinical, demographic, and multi-omics features—through a cross-attention mechanism and employs a Mixture-of-Experts (MoE) architecture to adaptively extract salient omics signals, thereby implicitly identifying patient-specific latent subgroups. This approach represents the first integration of multimodal embeddings and latent subgroup retrieval into counterfactual survival modeling, enabling simultaneous characterization of baseline survival dynamics and treatment-dependent effects. Evaluated on the METABRIC and TCGA-LUAD datasets, CURE significantly outperforms strong existing baselines in terms of time-dependent concordance index (C^td) and integrated Brier score (IBS).

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

This paper tackles the problem of time-to-event counterfactual survival prediction, aiming to optimize individualized survival outcomes in the presence of heterogeneity and censored data. We propose CURE, a framework that advances counterfactual survival modeling via comprehensive multimodal embedding and latent subgroup retrieval. CURE integrates clinical, paraclinical, demographic, and multi-omics information, which are aligned and fused through cross-attention mechanisms. Complex multi-omics signals can be adaptively refined using a mixture-of-experts architecture, emphasizing the most informative omics components. Building upon this representation, CURE implicitly retrieves patient-specific latent subgroups that capture both baseline survival dynamics and treatment-dependent variations. Experimental results on METABRIC and TCGA-LUAD datasets demonstrate that proposed CURE model consistently outperforms strong baselines in survival analysis, evaluated using the Time-dependent Concordance Index ($C^{td}$) and Integrated Brier Score (IBS). These findings highlight the potential of CURE to enhance multimodal understanding and serve as a foundation for future treatment recommendation models. All code and related resources are publicly available to facilitate the reproducibility https://github.com/L2R-UET/CURE.

Problem

• time-to-event
• counterfactual
• survival prediction
• heterogeneity
• censored data

Innovation

• counterfactual survival prediction
• multimodal fusion
• mixture-of-experts
• latent subgroup retrieval
• time-to-event modeling