Multireader receiver operating characteristic studies: a comparison of study designs

Area of Science:

• Medical Imaging Analysis
• Statistical Modeling in Healthcare

Background:

• Multireader receiver operating characteristic (ROC) studies are crucial for evaluating diagnostic test accuracy.
• The traditional "paired-case, paired-reader" design is commonly employed in these studies.

Purpose of the Study:

• To assess the statistical power of the traditional paired-case, paired-reader design.
• To compare the power of this design with alternative study designs for inferences on relative test accuracies.

Main Methods:

• Statistical power was computed using the noncentrality parameter of an F statistic.
• Power calculations considered reader and patient sample sizes, and variability and correlation between readings.

Main Results:

• The traditional design requires fewer verified cases for a fixed power and Type I error rate.
• A hybrid design, where readers interpret distinct patient samples, reduces reader and reading workload but increases the number of verified cases.

Conclusions:

• The optimal study design choice is contingent upon the specific tests being compared.
• Factors influencing design selection include resource limitations, prior knowledge of correlations and variability, and logistical complexity.