Development and Validation of a Multimodal-Multitask Deep Learning Approach for Estimating Late Distant Recurrence...

Late distant recurrence (DR) remains a persistent risk in hormone receptor-positive (HR+) early breast cancer after completion of 5 years of endocrine therapy (ET). We developed and validated a multimodal artificial intelligence (AI) model to improve long-term risk stratification and to explore heterogeneity in benefit from extended letrozole therapy (ELT). The deep learning model integrating digitized hematoxylin and eosin whole-slide images with clinicopathologic variables was developed using 2,271 patients from the National Surgical Adjuvant Breast and Bowel Project (NSABP) B-42 trial with five-fold cross-validation and externally validated in 4,300 patients from the TAILORx trial who were disease-free at 5 years from the initial diagnosis. Prognostic performance was evaluated using hazard ratios (HR) and absolute risk differences. Exploratory analyses assessed ELT benefit across model-defined risk groups. In NSABP B-42, the model stratified patients into groups with markedly different outcomes, with a 10-year absolute DR risk difference of 7.95% between high- and low-risk groups [HR, 5.71; 95% confidence interval (CI), 3.5-9.317; P < 0.001]. High-risk patients derived greater absolute benefit from ELT (4.09%) than low-risk patients (0.49%). External validation in the independent TAILORx cohort confirmed prognostic performance, with MI Clarity multimodal-multitask identifying patients with significantly different late DR outcomes (HR, 1.893; 95% CI, 1.413-2.534; P < 0.001). This multimodal AI approach using routine pathology and clinical data enables robust and generalizable stratification of late DR risk in HR+ breast cancer. This scalable strategy may complement existing genomic assays and support more individualized decisions about extended ET.

Late DR is a major cause of death in HR+ breast cancer, and deciding who needs longer hormone therapy is challenging. Current genomic tests are useful but have limitations. This study shows that an AI model using pathology and clinical data identifies patients at high or low risk of late recurrence. High-risk patients may benefit more from extended hormone therapy, offering an accessible way to guide treatment decisions.