Performance of tests based on the area under the ROC curve for multireader diagnostic data

One of the main objectives of disease prevention is to lower the healthcare costs and improve the quality of life. To achieve this, reliable diagnostic tools are needed. The diagnostic performance of a tool can be measured by the ROC curve and the AUC. However, some diagnostic tools such as MRI images are not objective, but depend on the interpretation of experts. Therefore, the accuracy of these tools may vary depending on who is interpreting them. To account for possible correlations when multiple readers collect data, Dorfman, Berbaum and Metz (1992) proposed using AUC pseudovalues from the jackknife sampling method and applying them to the mixed model to analyze the diagnostic reagent's accuracy. However, pseudovalues may go beyond the AUC range. Also, the random effect estimate may be negative due to a small number of readers. This paper develops tests based on AUC estimates and gives their asymptotic distribution. Moreover, a two-stage test is suggested to correct for negative random effect estimates. Four tests are created in total and their performance is evaluated by Monte Carlo simulations. The distributional assumption's robustness of these tests is checked, and their applicability is demonstrated by two real data sets.