Small sample estimation of relative accuracy for binary screening tests

Comparing the accuracy of two screening tests is ideally achieved by administering both tests as well as a gold standard test to all study subjects. In practice, a more ethical screen positive study design is often used, one that requires gold standard determination only for subjects that screen positive on either test under investigation. Although it is not possible to quantify the absolute accuracy of each test with such a design,the relative accuracy of the tests can be estimated. Since relative accuracy estimation has poor small sample properties, adjusted estimators based on adding constants to the observed data have been proposed. The adjusted estimators have the advantage that they yield point and variance estimates of relative accuracy in all settings. However, we show through both theory and numerical examples that adding constants to the data can be beneficial or detrimental to small sample performance. Furthermore, the performance of the adjusted estimator depends not only on the magnitude of the constant but also on parameters that cannot be estimated with data from a screen positive study, making selection of an optimal constant difficult in practice. We also examine the performance of the adjusted estimator using data from a study comparing the accuracy of two screening tests for cervical cancer.