The effect of two priors on Bayesian estimation of "Proper" binormal ROC curves from common and degenerate datasets

We showed previously that maximum-likelihood (ML) and Bayesian (with a flat prior on a common parameterization of the model) estimates of "proper" binormal receiver operating characteristic (ROC) curves produce similar results. We propose a new prior that is flat over the area under the ROC curve (AUC) and investigate its effect on the Bayesian estimates.

In two simulation studies, we compared Bayesian estimation of the AUC with the two prior probability distributions against ML estimation in terms of root mean squared error (RMSE) and the coverage of 95% confidence (or credible) intervals (both abbreviated CIs). In the first study, we simulated categorical data that tend to be "well-behaved" and produce ROC curve estimates that most would consider reasonable. In the second study, we simulated coarsely discretized categorical data that often included so-called degenerate datasets that cause the ML estimate to be the perfect ROC curve.

For the well-behaved datasets, all three AUC estimates were similar in terms of RMSE and 95% CI coverage. For the coarsely discretized datasets, the RMSE of ML was consistently greater than that of Bayesian estimation and the 95% CI coverage of ML estimation was consistently below nominal, whereas the 95% CI coverage of Bayesian estimation was consistently equal to, or greater than, nominal.

Bayesian estimation with a flat prior on the AUC can provide reasonable inference from datasets with coarsely categorized data that are prone to be degenerate and produce results similar to other estimation methods on well-behaved datasets.