The area under the ROC curve and its competitors

Area of Science:

• Medical Decision Making
• Biostatistics
• Diagnostic Test Evaluation

Background:

• The area under the receiver operating characteristic (ROC) curve is widely used to assess diagnostic test performance.
• However, the area under the ROC curve (AUC) can be an inconsistent metric, as tests with similar clinical impact may yield different AUC values.

Purpose of the Study:

• To address the inconsistency of the area under the ROC curve (AUC) as a diagnostic test evaluation criterion.
• To propose a new class of diagnosticity measures (DMs) that are consistent and optimal.
• To demonstrate the utility of these DMs in medical decision-making contexts.

Main Methods:

• Introduced a class of diagnosticity measures (DMs) based on regret theory.
• Developed methods to calculate DMs from the receiver operating characteristic (ROC) curve configuration.
• Defined two scaled variants of the ROC curve to aid in the analysis.

Main Results:

• Demonstrated that the area under the ROC curve (AUC) is an inconsistent measure of diagnostic test power.
• Proposed a class of diagnosticity measures (DMs) that are uniquely defined and optimally chosen once a regret measure is specified.
• Showed that these DMs are calculable from the ROC curve, offering a consistent alternative.

Conclusions:

• The proposed diagnosticity measures (DMs) provide a more consistent and theoretically sound approach to evaluating diagnostic tests compared to the area under the ROC curve (AUC).
• These DMs are adaptable based on the chosen measure of diagnostic uncertainty, offering flexibility in application.
• The methods and scaled ROC variants introduced can benefit students and researchers in medical decision making and diagnostic test evaluation.