A Comprehensive Review of Performance Metrics for Computer-Aided Detection Systems

This paper aims to provide a structured analysis of the performance metrics used in computer-aided detection (CAD) systems, specifically focusing on lung nodule detection in computed tomography (CT) images. By examining key metrics along with their respective strengths and limitations, this study offers guidelines to assist in selecting appropriate metrics. Evaluation methods for CAD systems for lung nodule detection are primarily categorized into per-scan and per-nodule approaches. For per-scan analysis, a key metric is the area under the receiver operating characteristic (ROC) curve (AUROC), which evaluates the ability of the system to distinguish between scans with and without nodules. For per-nodule analysis, the nodule-level sensitivity at fixed false positives per scan is often used, supplemented by the free-response receiver operating characteristic (FROC) curve and the competition performance metric (CPM). However, the CPM does not provide normalized scores because it theoretically ranges from zero to infinity and largely varies depending on the characteristics of the data. To address the advantages and limitations of ROC and FROC curves, an alternative FROC (AFROC) was introduced to combine the strengths of both per-scan and per-nodule analyses. This paper discusses the principles of each metric and their relative strengths, providing insights into their clinical implications and practical utility.