Multimodal Transformer Integrating CT and Pathology for Postoperative Risk Stratification in Stage I Non-Small Cell...

Purpose To develop and evaluate an interpretable, multimodal deep learning model to improve postoperative risk stratification and predict adjuvant therapy benefit in stage I non-small cell lung cancer (NSCLC). Materials and Methods In this multicenter retrospective study of patients with stage I NSCLC, a multimodal model called the Deep Learning-Radiomic-Text Transformer (DLRTransformer) was developed by integrating multiview 2.5-dimensional deep learning features from CT scans, radiomics, clinical variables, and prognostic embeddings extracted from pathology reports using a large language model. A central transformer architecture fuses these features to predict disease-free survival. Model performance was evaluated on independent external datasets and compared with guideline-based risk stratification. Associations between model-derived risk scores and tumor microenvironment features were analyzed to assess biologic interpretability. Results A total of 1073 patients (mean age ± SD, 60.8 years ± 9.8; 550 male) were enrolled from five centers. On external datasets, DLRTransformer demonstrated robust prognostic performance (concordance index, 0.77-0.78), outperforming the guideline-based risk stratification (concordance index, 0.60-0.65). Risk scores were associated with tumor microenvironment phenotypes, including increased neoplastic cellularity and reduced tertiary lymphoid structures. A treatment-by-risk interaction was observed (interaction hazard ratio [HR], 0.14; 95% CI: 0.06, 0.32; P < .01). In propensity score-matched subcohorts, adjuvant therapy improved disease-free survival in high-risk patients (HR, 0.30; 95% CI: 0.11, 0.84; P = .02) but not in low-risk patients (HR, 1.70; 95% CI: 0.97, 2.99; P = .06). Conclusion The DLRTransformer improved postoperative risk stratification compared with guideline-based risk stratification and identified patients with stage I NSCLC who benefited from adjuvant therapy. Keywords: Deep Learning, Large Language Models LLMs, CT, Prognosis & Prediction, Non-Small Cell Lung Cancer (NSCLC), Multimodal Deep Learning, Risk Stratification, Adjuvant Therapy, Tumor Microenvironment (TME) Supplemental material is available for this article. © The Author(s) 2026. Published by the Radiological Society of North America under a CC BY 4.0 license.