On the shape of the population ROC curve

Human observers often do not produce empirical operating points near the northeast corner of the receiver operating characteristic (ROC) plot, and thus the local shape of the population ROC curve is unknown.

We call attention to occult abnormalities and propose that considerations by human observers of the prior probability of occult abnormalities can cause the shape of the local population ROC curve to be convex, a straight line, or concave near the northeast corner of the ROC plot. We further conducted a set of simulated detection-task (without-search) experiments with human observers and, mathematically, with an ideal observer and a model observer. In the experiments, we used signals, pseudo-signals that were similar to signals, and random image noise. The relative frequency of occult signals was controlled in the experiments.

In the simulated experiments, the population ROC curve of the ideal observer was always convex, but those of the model observer and of human observers were convex, a straight-line, or concave, depending on the relative frequency of occult signals. The population ROC curve for the model observer was identical to that for the ideal observer when knowing the relative frequency of occult signals was not important for the ideal observer, and it was similar to that for human observers otherwise.

Observer consideration of the prior probability of occult abnormalities is important in ROC studies and could cause unexpected shapes of the local population ROC curve. Absence of empirical operating points near the northeast corner of the ROC plot may be caused by occult abnormalities.