Related Experiment Video
Updated: Mar 3, 2026
Author Spotlight: A 3D Digital Model for the Diagnosis and Treatment of Pulmonary Nodules
Published on: May 19, 2023
Deep learning-based pulmonary nodule risk assessment outperforms established malignancy risk scores in lung cancer
Eduardo J Mortani Barbosa1, Yohan Kim1, Yanbo Zhang2
1Department of Radiology, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA 19104, United States.
Background:
Pulmonary nodules are commonly encountered in lung cancer screening. The risk of malignancy varies widely and is generally estimated using expert consensus guidelines (Lung CT Imaging Reporting and Data Systems [Lung-RADS]).
Purpose:
To assess the performance of a deep learning algorithm (Deep Pulmonary Nodule Profiler [DeepPNP]) for pulmonary nodule malignancy risk estimation in a lung cancer screening dataset and the effect of data enrichment in model training.
Materials And Methods:
A retrospective analysis was conducted using 3 datasets. DeepPNP is a 3D convolutional network (EfficientNet-B0-based) operating on nodule-centered 3D patches. For the DeepPNP model training and validation, the National Lung Screening Trial (NLST) dataset was combined with 2 independent malignant nodule-only datasets, resulting in a merged dataset of 28 057 nodules, including 2362 malignant nodules. An ablation model (DeepPNP-NLST) was trained on NLST only. The testing was conducted on a held-out dataset from the NLST dataset. Performance metrics, including sensitivity, specificity, precision, F1 score, and accuracy, were analyzed across 3 operating thresholds selected based on specificities of 0.80, 0.85, and 0.90 (selected on the validation set). Benchmarks included Lung-RADS v2022 and the PanCan model.
Results:
On the NLST test set (including 2597 nodules from 1243 CT scans), DeepPNP achieved an area under the receiver operating characteristic curve (ROC AUC) of 0.96 (95% confidence interval [CI], 0.95-0.97), outperforming Lung-RADS AUC = 0.91 (95% CI, 0.89-0.94; P < .001) and PanCan AUC = 0.93 (95% CI, 0.91-0.95; P < .001). DeepPNP-NLST had an AUC of 0.95 (95% CI, 0.93-0.97; P = .045 vs DeepPNP), indicating a modest gain from positive-only supplementation. Subgroup analyses showed consistent outperformance across nodule sizes and types. Operating-point metrics at 0.80/0.85/0.90 specificity are reported; at 0.80 specificity, DeepPNP achieved sensitivity of 0.94 (100/107; 95% CI, 0.88-0.98) and specificity of 0.88 (2196/2490; 95% CI, 0.87-0.90).
Conclusion:
DeepPNP outperformed established malignancy risk models in lung cancer screening. The inclusion of biopsy-confirmed malignant nodules from 2 external datasets provided a measurable performance gain, underscoring the importance of data enrichment during model training.
