A tutorial on the use of ROC analysis for computer-aided diagnostic systems.

Area of Science:

• Medical Imaging and Diagnostics
• Biostatistics
• Machine Learning in Healthcare

Background:

• Computer-aided diagnostic (CAD) systems rely on classification performance evaluation.
• Traditional methods often use threshold-dependent measures (e.g., sensitivity, specificity).
• Existing literature lacks practical guidance for real-world CAD system development, especially with non-normally distributed data.

Purpose of the Study:

• To review the application of receiver operating characteristic (ROC) curves in CAD systems.
• To present a statistical framework for evaluating CAD system classification performance.
• To bridge the gap between theoretical ROC analysis and practical application with real-world data.

Main Methods:

• Review of threshold-specific performance measures (sensitivity, specificity).
• Detailed examination of the threshold-independent measure: area under the ROC curve (AUC).
• Derivation of methods for evaluating classification performance in CAD systems, particularly for ultrasonic tissue characterization.

Main Results:

• Threshold-independent performance measures, such as AUC, offer more representative classification results for CAD systems.
• The study provides a framework applicable to real-world data, addressing limitations of theoretical ROC analysis.
• Guidance is offered for integrating ROC analysis algorithms into classification systems.

Conclusions:

• Area under the ROC curve is a superior, threshold-independent metric for evaluating CAD system performance.
• The presented framework and information are vital for scientists developing and implementing CAD systems.
• Understanding ROC analysis principles is essential for practical, effective CAD system development and data interpretation.