Lightweight hybrid foundation model for lung cancer prognosis based on low-dose chest X-ray images

Lung cancer (LC) is the leading cause of cancer mortality worldwide. Accurate prognosis in vulnerable populations, particularly in low-resource settings remains challenging with high radiation dose of standard imaging modalities. While chest X-rays (CXRs) are safer and more accessible, their low spatial resolution limits prognostic accuracy. Existing multimodal models primarily rely on computationally intensive architectures, such as computed tomography (CT) or positron emission tomography (PET) inputs, which reduce their clinical utility. This study aimed to develop and validate a lightweight hybrid foundation model (LHFM) that integrates CXR imaging with clinical data to enable accurate LC prognosis in low-resource settings.

We developed a LHFM that integrates visual features from CXR images extracted by a segment anything model (SAM)-Med2D encoder with semantically enriched prompts generated by BioGPT and clinical metadata. These multimodal features are fused via a dual-branch transformer architecture for survival prediction. The model was trained and validated on the JSRT and PadChest datasets, with external validation on multicenter datasets including the NIH CXR. Performance was evaluated using the concordance index (C-index), area under the receiver operating characteristic curve (AUROC), and Kaplan-Meier (KM) survival analysis.

The proposed LHFM achieved superior prognostic performance, with a C-index of 0.910 [95% confidence interval (CI): 0.898-0.922, standard deviation (SD) =0.006] and AUROC of 0.935 (95% CI: 0.927-0.943, SD =0.004), outperforming existing multimodal benchmarks (P<0.001). KM curves demonstrated significant separation between the high-risk and low-risk groups. Domain-shift robustness testing across heterogeneous external datasets demonstrated representation stability under distribution shift.

LHFM establishes a new paradigm for prognostic precision by exhibiting significant performance from low-dose CXR. This hybrid approach directly addresses the implementation gap in clinical artificial intelligence (AI), offering a scalable, equitable, and immediately applicable solution for personalized cancer care in resource-limited and radiography first workflows, with potential applicability across other cancer types.