Comparison of interlaboratory survey date in terms of receiver operating characteristic (ROC) indices

Receiver Operating Characteristic (ROC) analysis in the field of visual imaging has hitherto dealt with the performance of imaging modalities, system displays, or individual readers. This study combines previously established interlaboratory survey techniques with ROC concepts to evaluate selected indices of group discrimination and of group decision criteria in a quantitative and comparative manner. One hundred forty-two subscribers to the College of American Pathologists Phantom-Imaging Series performed a series of visual detection tasks on a fabricated radioactive phantom image. Results suggest that group performance may be legitimately assessed by ROC analysis if the variables of system modality, image display, and cases are controlled, and the number of trials is sufficiently large. The latter condition may be fully satisfied through interlaboratory survey techniques that utilize currently existing peer groups.