Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

2026-08-07

AI Summary

Current approaches to predicting pathological complete response (pCR) in neoadjuvant chemotherapy for breast cancer predominantly rely on imaging from a single timepoint, limiting their ability to capture dynamic treatment-related changes. This work proposes the first longitudinal analysis framework that integrates a 3D foundation model with temporal dynamics modeling. Specifically, it employs a frozen pretrained 3D encoder, Pillar-0, to extract features from dynamic contrast-enhanced MRI (DCE-MRI) at each of four clinical timepoints, combined with a temporal dynamics network (TDN), time-aware voxel embeddings, and a multimodal fusion strategy that incorporates both imaging and clinical data. Evaluated on a cohort of 982 patients, the proposed method significantly outperforms single-timepoint and unimodal baselines, achieving an AUROC of 73.6% and a balanced accuracy of 69.1%, thereby establishing a new paradigm for longitudinal, multimodal pCR prediction.

Abstract

Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging during treatment could support treatment response assessment. Many existing imaging-based approaches rely on a single static timepoint, which fails to capture changes that occur during treatment. In this work, we present a longitudinal framework that combines a frozen 3D foundation encoder (Pillar-0) with our Temporal Dynamics Network (TDN) to predict treatment response from serial Dynamic Contrast-Enhanced (DCE) MRI acquired across four clinical timepoints from pre-treatment to pre-surgery. The TDN combines time-aware volumetric embeddings with clinical and treatment data to predict pCR. Evaluated on 982 patients from the combined I-SPY2 and ACRIN-6698 cohort, the proposed model achieves strong performance across all reported metrics when longitudinal 3D imaging is fused with clinical data (test AUROC: 73.6%, balanced accuracy: 69.1%). While clinical variables provide the strongest individual predictive signal, longitudinal 3D imaging contributes complementary information when fused with clinical data, improving pCR prediction. Our source code is available at: https://github.com/omarftt/longitudinal_temporal_pillar.

Problem

• pathologic complete response
• neoadjuvant chemotherapy
• longitudinal imaging
• DCE-MRI
• breast cancer

Innovation

• longitudinal modeling
• 3D foundation model
• temporal dynamics network
• DCE-MRI
• pCR prediction