Related Experiment Videos
Multireader, multicase receiver operating characteristic methodology: a bootstrap analysis
D D Dofrman1, K S Berbaum, R V Lenth
1Department of Psychology, University of Iowa, Iowa City 52242, USA.
Rationale And Objectives:
We evaluated by bootstrapping the conclusions obtained by the Dorfman-Berbaum-Metz (DBM) receiver operating characteristic (ROC) method and by the Toledano-Gatsonis (TG) method on a well-known data set.
Methods:
We bootstrapped in two ways, resampled cases while holding readers fixed and resampled both cases and readers.
Results:
When an analysis of variance of pseudovalues implies that reader variance and all random interactions with treatment are essentially zero, then case-resampling bootstrap and the DBM and TG methods should give the same results. Case-resampling bootstrap and the DBM and TG methods did give highly similar results for both individual readers and the averages over all readers. Both the case-resampling bootstrap and the reader-case resampling bootstrap gave smaller standard errors for group than for individual reader means, thereby providing evidence for a trade-off of readers and cases with regard to precision and power in this data set.
Conclusion:
Case-resampling bootstrap provides some justification for the DBM and TG methods.