Diagnostic performance with sensitivity/specificity optimized computer-aided detection to detect common abnormalities...

To evaluate how two fixed computer-aided detection (CAD) operating modes-sensitivity-optimized (SE-CAD) and specificity-optimized (SP-CAD)-affect radiologists' diagnostic performance in chest radiography (CXR). Previous studies have focused on the performance of standalone CAD at different threshold settings. However, in clinical practice, CAD is commonly used as a decision-support tool for radiologists. This study aimed to evaluate the impact of different CAD threshold settings on radiologists' diagnostic performance.

This retrospective single-center diagnostic accuracy study evaluated posteroanterior or anteroposterior CXRs acquired between 2013 and 2023, interpreted with and without AI assistance, using CT as the reference standard. Six radiologists (three residents / three board-certified) read all cases four times over two sessions: first without CAD, then with either SE_CAD or SP_CAD, and after 1-month washout period, second session using the alternative CAD.

375 patients (mean age, 65 ± 15 years; 170 women) were included. The dataset comprised 139 cases with pulmonary opacities (nodules / masses / consolidations), 119 with pleural effusion, 29 with pneumothorax, and 161 normal cases, including overlaps. SP_CAD-assisted readings showed higher specificity than SE_CAD-assisted for pulmonary opacities (p < 0.001). Sensitivity did not differ significantly between SE_CAD and SP_CAD assistance. For pulmonary opacities, CAD-assisted reading showed higher sensitivity (both CAD: p < 0.001) but lower specificity (SE_CAD: p = 0.006; SP_CAD: p = 0.03) than unassisted reading. For pleural effusion, CAD-assisted improved sensitivity (SE_CAD: p < 0.001; SP_CAD: p = 0.02) and specificity (both CAD: p < 0.001) than unassisted reading. For pneumothorax, SE_CAD assistance did not significantly affect sensitivity but decreased specificity, whereas SP_CAD assistance improved sensitivity without significantly affecting specificity.

The specificity-prioritized CAD showed higher specificity than the sensitivity-prioritized CAD, without a significant reduction in sensitivity. These findings suggest that different CAD operating points can influence radiologists' diagnostic performance. We compared radiologist performance using sensitivity-optimized and specificity-optimized computer-aided detection (CAD). Different predefined CAD operating points influenced radiologist performance, resulting in differences in specificity for nodules/masses/consolidations. Our results show that different preset decision thresholds within the same AI model can influence diagnostic performance in AI-assisted radiograph interpretation.