Externally Tested AI Models for Malignancy Classification of Lung Nodules at CT: A Systematic Review and...

Purpose To evaluate the pooled diagnostic accuracy of externally tested artificial intelligence (AI) models for malignancy classification of lung nodules at chest CT. Materials and Methods A systematic search of PubMed, Embase, Web of Science, Cumulative Index to Nursing and Allied Health Literature, and the Cochrane Library was performed in March 2023 and updated on January 12, 2025, to identify studies evaluating AI models for malignancy classification of lung nodules at chest CT using pathology and/or at least 2-year follow-up as reference standards. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool, and pooled sensitivity and specificity were estimated using bivariate random-effects models. Results Twenty-one studies including 7454 nodules were analyzed, with lung cancer prevalence ranging from 5.7% (17 of 297) to 91.5% (214 of 234). All models were based on deep learning; 17 of the 21 studies (81%) involved Asian populations, 16 (76%) used nonscreening populations, 14 (67%) reported two-dimensional or three-dimensional convolutional neural network (CNN) architectures, and eight (38%) specified predefined malignancy thresholds. High risk of bias was identified in five studies for patient selection and in two for index testing. Pooled sensitivity was 88%, pooled specificity was 75%, positive likelihood ratio was 3.55, negative likelihood ratio was 0.16, area under the receiver operating characteristic curve was 0.89, and the diagnostic odds ratio was 22.4. Heterogeneity was high (I² > 90%). Model architecture was associated with specificity, with higher values in studies reporting two-dimensional or three-dimensional CNNs compared with those without reported architecture (82%-83% vs 58%, P = .03; meta-regression P = .02); other subgroup analyses showed no evidence of differences. Conclusion Externally tested AI models demonstrated high sensitivity but moderate specificity for malignancy classification of lung nodules at chest CT, supporting a potential role in rule-out strategies. However, substantial heterogeneity, inconsistent reporting, and risk of bias limit interpretation. Keywords: Lung Cancer, Artificial Intelligence, Malignancy Classification, External Validation, Meta-Analysis Supplemental material is available for this article. © RSNA, 2026 See also commentary by Bressem and Kim in this issue.