Evaluating diagnostic tests: The area under the ROC curve and the balance of errors

Area of Science:

• Medical diagnostics
• Biostatistics
• Health informatics

Background:

• Accurate medical diagnosis is fundamental to patient care.
• Evaluating the performance of diagnostic tests is essential.
• The area under the receiver operating characteristic (ROC) curve (AUC) is a common accuracy measure.

Purpose of the Study:

• To explore the fundamental weaknesses of the AUC measure, particularly its inability to effectively balance different types of misdiagnoses.
• To introduce and describe an alternative measure, the H measure, for evaluating diagnostic test accuracy.

Main Methods:

• Analysis of the inherent properties of the AUC measure.
• Exploration of scenarios where ROC curves cross, highlighting AUC's limitations.
• Description of the proposed H measure as an alternative to AUC.

Main Results:

• The AUC measure has a fundamental weakness in balancing different kinds of misdiagnoses.
• This limitation is an inherent property of the AUC's definition, not just an arbitrary choice.
• The H measure is presented as a potential alternative that addresses this weakness.

Conclusions:

• The AUC measure, despite its widespread use, has significant limitations in accurately reflecting diagnostic test performance due to its handling of misdiagnoses.
• The proposed H measure offers a more effective approach to balancing different types of diagnostic errors.
• Further investigation into the H measure is warranted for improved diagnostic test evaluation.