TEMPO: Transformers for Temporal Disease Progression from Cross-Sectional Data

Event-Based Models (EBMs) infer biomarker progression from cross-sectional data but typically only as ordinal sequences and rely on rigid model assumptions. We propose $\text{textsc}Tempo$, a Transformer architecture that learns both ordinal and continuous event sequences through simulation-based supervised learning. $\text{textsc}Tempo$ uses two Transformer modules: one treats biomarkers as tokens to infer event sequencing; the other treats patients as tokens, representing each by their per-biomarker abnormality profile, to infer patients’ disease stages. On synthetic benchmarks, $\text{textsc}Tempo$ reduces normalized Kendall’s Tau distance by 52.89% and staging MAE by 25.33% compared to state-of-the-art SA-EBM, with larger reductions in high-dimensional settings (58.88% and 61.10%). Applied to ADNI, $\text{textsc}Tempo$ recovers a biologically plausible Alzheimer’s progression: early medial temporal atrophy, followed by amyloid accumulation and cognitive decline, and late-stage tau pathology with terminal acceleration of global neurodegeneration – broadly consistent with established disease models. $\text{textsc}Tempo$ also eliminates the need to derive custom inference algorithms and enables rapid empirical comparison of generative hypotheses.