A note on the effect of the intraclass correlation in the multiple reading procedure with a unanimity rule

The use of multiple reading procedures to improve the performance of a diagnostic test occurs often in practice. Evaluation of the utility of multiple reading procedures, however, usually ignores the effect of the intraclass correlation. This paper provides a quantitative assessment of this effect in the multiple reading procedure with a unanimity rule with respect to sensitivity, specificity, positive and negative predictive values. We have found that when the disease prevalence is rare or moderate (less than or equal to 0.20), use of the multiple reading procedure with a unanimity rule is effective in increasing the positive predictive value of a single reading procedure for the situation in which the variation of responses among different subjects and the intraclass correlation among repeated tests are small. This is, however, not true for the situation in which the disease is rare and the variation of responses among different subjects is large, even when the intraclass correlation is small or 0. Furthermore, when the disease is rare and the variation of responses among subjects is small, a small or moderate intraclass correlation can substantially decrease the positive predictive value that one calculates under the assumption that the intraclass correlation is equal to 0. In general, when the disease is rare or moderate (less than or equal to 0.20), the intraclass correlation between repeated tests and the variation of responses among subjects have little effect on the negative predictive value.