Evaluating the diagnostic performance of artificial intelligence-assisted decision-making software for pulmonary...

Emerging evidence suggested that artificial intelligence (AI) may offer particular benefits in resource-limited clinical settings with high patient loads and constrained radiology expertise. The present study aimed to evaluate the diagnostic performance of an AI-assisted decision-making software (DMS) for pulmonary nodules detected on computed tomography (CT) among physicians in a resource-limited clinical setting.

In this retrospective multi-reader, multi-case study, three pulmonologists and three radiologists from a secondary hospital independently assessed 200 enriched chest CT scans with and without AI-assisted DMS. The dataset was balanced with 100 benign and 100 malignant nodules to provide a consistent challenge for both physicians and the AI system. Diagnostic performance was measured by comparing the average area under the receiver operating characteristic curves (AUC) with and without AI support. Sensitivity and specificity were evaluated at the 5% and 65% malignancy thresholds, and inter-reader agreement on disease management plans was examined.

AI-assisted DMS significantly improved readers' diagnostic performance, with the average AUC increasing from 0.78 to 0.89 (mean difference: 0.11, 95% confidence interval [CI]: 0.08, 0.14). Improvements were consistent across readers' experience levels and specialties. Sensitivity at the 5% malignancy threshold reached 97.3% (95% CI: 95.1%, 99.6%) with AI assistance, while specificity improved by 18.5% (95% CI: 6.5%, 30.5%). At the 65% threshold, sensitivity and specificity increased by 21.2% and 7.8%, respectively. In addition, the overall inter-reader agreement enhanced from 0.19 to 0.40 (p < 0.01), although agreement on non-surgical diagnostic procedures remained relatively lower compared to other categories.

AI-assisted DMS showed great potential in improving diagnostic performance for CT pulmonary nodule management in the resource-limited setting. Strengthening referral pathways for intermediate-risk cases might further support appropriate clinical decision-making and help align patient evaluation with available expertise. Continued prospective real-world studies with longitudinal follow-up and histopathological confirmation would contribute to expanding the evidence base and guiding its broader integration into routine clinical practice.