Evaluation of diagnostic accuracy in free-response detection-localization tasks using ROC tools

Area of Science:

• Medical imaging analysis
• Diagnostic performance evaluation
• Radiology research

Background:

• Free-response studies are crucial for evaluating diagnostic systems detecting multiple lesions (e.g., CT colonoscopy polyps).
• Traditional receiver operating characteristic (ROC) analysis is insufficient for free-response data, necessitating free-response ROC (FROC) methodology.
• Current FROC methods are cumbersome, and alternative ROI-ROC approaches require complex region delineation.

Purpose of the Study:

• To propose a novel approach for analyzing FROC data using conventional ROC tools.
• To simplify the evaluation of diagnostic systems without requiring ROI delineation or data reduction.
• To calibrate FROC and ROC curves conceptually and numerically.

Main Methods:

• Utilizing FROC study design parameters to enable analysis with standard ROC tools.
• Developing a method that avoids physical delineation of regions of interest (ROIs).
• Comparing the proposed approach with traditional nonparametric FROC methods using data from a multi-reader colon cancer detection study.

Main Results:

• The proposed approach allows FROC data to be analyzed using conventional ROC tools.
• No explicit ROI delineation or data reduction is necessary.
• Performance index differences between the new approach and nonparametric FROC are asymptotically negligible and practically small.

Conclusions:

• The novel approach simplifies FROC data analysis by leveraging standard ROC tools.
• This method offers a more accessible and efficient way to evaluate diagnostic systems for multiple lesion detection.
• The findings are validated using a large-scale colon cancer detection study, demonstrating practical applicability.