TEMPO: Transformers for Temporal Disease Progression from Cross-Sectional Data

2026-04-25

AI Summary

Existing methods struggle to accurately model disease progression from cross-sectional data, often relying on strong assumptions or supporting only ordinal representations. This work proposes TEMPO, the first framework to employ Transformers for modeling disease progression, featuring a dual-module architecture that separately handles biomarker ordering and patient staging while unifying ordinal and continuous event sequences through simulation-based supervised learning. TEMPO requires no custom inference algorithms and enables flexible generation and comparison of progression hypotheses. On synthetic data, TEMPO reduces the normalized Kendall’s Tau distance by 52.89% and staging MAE by 25.33% compared to SA-EBM, with even greater improvements in high-dimensional settings. Applied to ADNI data, it successfully recovers an Alzheimer’s disease progression trajectory consistent with established biological consensus.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Event-Based Models (EBMs) infer biomarker progression from cross-sectional data but typically only as ordinal sequences and rely on rigid model assumptions. We propose \textsc{Tempo}, a Transformer architecture that learns both ordinal and continuous event sequences through simulation-based supervised learning. \textsc{Tempo} uses two Transformer modules: one treats biomarkers as tokens to infer event sequencing; the other treats patients as tokens, representing each by their per-biomarker abnormality profile, to infer patients' disease stages. On synthetic benchmarks, \textsc{Tempo} reduces normalized Kendall's Tau distance by 52.89\% and staging MAE by 25.33\% compared to state-of-the-art SA-EBM, with larger reductions in high-dimensional settings (58.88\% and 61.10\%). Applied to ADNI, \textsc{Tempo} recovers a biologically plausible Alzheimer's progression: early medial temporal atrophy, followed by amyloid accumulation and cognitive decline, and late-stage tau pathology with terminal acceleration of global neurodegeneration -- broadly consistent with established disease models. \textsc{Tempo} also eliminates the need to derive custom inference algorithms and enables rapid empirical comparison of generative hypotheses.

Problem

• Temporal Disease Progression
• Cross-Sectional Data
• Event-Based Models
• Biomarker Sequencing
• Disease Staging

Innovation

• Transformers
• Temporal Disease Progression
• Cross-Sectional Data
• Event-Based Modeling
• Simulation-Based Learning