Dynamic Survival Prediction using Longitudinal Images based on Transformer

2025-08-12

AI Summary

Existing longitudinal medical imaging survival analysis methods suffer from three key limitations: insufficient utilization of censored data, neglect of temporal dependencies among multi-timepoint images, and poor interpretability. To address these, we propose SurLonFormer—a novel framework that deeply integrates the Transformer architecture with the Cox proportional hazards model. It employs a vision encoder to extract spatial features and a sequence encoder to capture temporal dynamics, while incorporating masked sensitivity analysis for dynamic risk prediction and biomarker localization. SurLonFormer jointly models censoring and longitudinal structure, enabling individualized, time-varying risk inference. Evaluated on a real-world Alzheimer’s disease cohort, SurLonFormer achieves statistically significant performance gains (C-index increase of 0.042, *p* < 0.01) and identifies interpretable, disease-progression–associated imaging biomarkers—demonstrating both clinical utility and mechanistic interpretability.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Machine Learning: Life-Long and Continual Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Survival analysis utilizing multiple longitudinal medical images plays a pivotal role in the early detection and prognosis of diseases by providing insight beyond single-image evaluations. However, current methodologies often inadequately utilize censored data, overlook correlations among longitudinal images measured over multiple time points, and lack interpretability. We introduce SurLonFormer, a novel Transformer-based neural network that integrates longitudinal medical imaging with structured data for survival prediction. Our architecture comprises three key components: a Vision Encoder for extracting spatial features, a Sequence Encoder for aggregating temporal information, and a Survival Encoder based on the Cox proportional hazards model. This framework effectively incorporates censored data, addresses scalability issues, and enhances interpretability through occlusion sensitivity analysis and dynamic survival prediction. Extensive simulations and a real-world application in Alzheimer's disease analysis demonstrate that SurLonFormer achieves superior predictive performance and successfully identifies disease-related imaging biomarkers.

Problem

• Improves survival prediction using longitudinal medical images
• Addresses censored data and temporal correlation limitations
• Enhances interpretability and identifies disease biomarkers

Innovation

• Transformer-based network for survival prediction
• Integrates longitudinal images and structured data
• Uses occlusion sensitivity for interpretability