Related Experiment Video
Updated: Jun 20, 2026
A 3D Digital Model for the Diagnosis and Treatment of Pulmonary Nodules
Published on: May 19, 2023
Comparison of three commercial AI tools for detection and malignancy assessment of incidental lung nodules
Oguzhan Tokur1,2,3,4, Oyunbileg von Stackelberg1,2,3, Luca Dulz1,2,3
1Department of Diagnostic and Interventional Radiology, Heidelberg University Hospital, Heidelberg, Germany.
Objectives:
Incidental pulmonary nodules are common on computed tomography (CT), and management typically relies on size and volume. Artificial intelligence (AI)-based tools show promise in nodule detection and risk assessment, but their clinical utility remains uncertain. This study aimed to evaluate the accuracy of AI-based software in detecting and predicting the malignancy of incidental pulmonary nodules.
Materials And Methods:
This retrospective study included patients selected from a cohort of 1,138 individuals who underwent chest CT between 2015 and 2024 and met the inclusion criteria. Patients were classified into benign and malignant groups. Malignancy was determined by histopathology or by at least 2 years of follow-up. Nodule location, size, and type were assessed using by both using radiology reports and AI tools. Three commercial tools (AI-I, AI-II, and AI-III) were assessed for nodule detection and malignancy risk prediction. Agreement was assessed using Cohen's kappa and the intraclass correlation coefficient (ICC), and diagnostic performance was evaluated using receiver operating characteristic analysis.
Results:
Chest CT scans of 374 patients (mean age 66 ± 9 yr.; range 37-88 yr.; 231 males) with at least one solid or part-solid nodule were evaluated. AI-I and AI-II demonstrated excellent agreement with radiology reports for nodule localization (κ = 0.95, p < 0.001) and moderate agreement for nodule type (κ = 0.46, p < 0.001). Size assessment showed excellent agreement with ICC values of 0.93 [95%CI = 0.92-0.94] for AI-I and 0.89 [95%CI = 0.86-0.91] for AI-II. AI-II differed from AI-III in malignancy prediction with AUC = 0.77 [95%CI = 0.72-0.81] and 0.89 [95%CI = 0.85-0.92], respectively. Additionally, AI-II showed significantly lower PPV (65.57% vs. 87.20%, p < 0.001) and accuracy (72.1% vs. 82%, p < 0.001) than AI-III.
Conclusion:
AI-based tools demonstrated high accuracy for incidental pulmonary detection; however, their performance in malignancy risk stratification differed substantially.
