Estimating the Area Under ROC Curve When the Fitted Binormal Curves Demonstrate Improper Shape

Area of Science:

• Medical diagnostics
• Statistical modeling
• Receiver Operating Characteristic (ROC) analysis

Background:

• The binormal model is standard for parametric ROC analysis.
• Fitted binormal ROC curves can exhibit unrealistic "improper" (non-concave) shapes.
• The impact of improperness on area under the ROC curve (AUC) estimates is investigated.

Purpose of the Study:

• To assess the reliability of AUC estimates from fitted binormal ROC curves when the underlying data generate a proper curve.
• To quantify the effect of severe improperness on AUC bias and confidence interval coverage.

Main Methods:

• Designed ROC scenarios to induce severe improperness in fitted binormal curves.
• Employed maximum likelihood estimation for fitting binormal curves.
• Conducted simulations to evaluate AUC bias, confidence interval coverage, and compare with non-parametric methods.

Main Results:

• Up to 96% of fitted curves showed severe improperness.
• Binormal AUC estimates exhibited minimal bias with near-nominal confidence interval coverage.
• Partial AUC estimates showed bias at high and low specificity ranges.

Conclusions:

• Severe improperness in fitted binormal ROC curves does not inherently compromise the reliability of overall AUC estimates.
• Sufficiently distributed data points mitigate the negative impact of improper curve shapes on AUC inferences.