Distribution-free confidence bounds for ROC curves

ROC curves are widely used for the evaluation of diagnostic tests to decide between "healthy" and "diseased" individuals when the measurements are on a continuous scale. These curves are graphical displays of the interdependence between specificity and sensitivity of the test varying with the cut-off point chosen for the decision. Up to now only point estimators derived from the empirical distribution functions are used which may be misleading if they are based on rather small samples. In this paper we propose reasonable confidence bounds for ROC curves and a corresponding point estimator. Our bounds are strongly related to two-sided distribution-free tolerance regions because they are constructed from minimum and maximum coverages which at a given value chi can be guaranteed with a confidence (2 pi*-1). The interpretation of the bounds is that if a cut-off-point is chosen on the basis of the ROC curve then with a nominal confidence of at least (2 pi*-1)2 the real sensitivity and specificity will be within a rectangle.