Dorfman-Berbaum-Metz method for statistical analysis of multireader, multimodality receiver operating characteristic...

Area of Science:

• Medical imaging analysis
• Statistical methods in diagnostics
• Receiver Operating Characteristic (ROC) curve analysis

Background:

• The Dorfman-Berbaum-Metz (DBM) method is used for analyzing multireader, multimodality receiver operating characteristic (ROC) data.
• Understanding the relationship between critical P value (alpha) and empirical type I error rate is crucial for reliable statistical testing.

Purpose of the Study:

• To examine the relationship between the critical P value (alpha) and the empirical type I error rate using the DBM method.
• To assess the robustness of the DBM method for variance analysis in ROC data.

Main Methods:

• A linear mixed-effect model was developed to simulate ROC data with multiple sources of variation.
• Experiments included varying numbers of readers, modalities, and case sample sizes.
• Three binormal population ROC curves with different areas were simulated.
• 2,000 datasets were analyzed for each experiment to determine the proportion of statistically significant modality differences at an alpha level of .05.

Main Results:

• The DBM method performed well for low and intermediate ROC curves, even with small sample sizes.
• For high ROC curves, results were conservative with small samples.
• No correlation was found between the observed type I error rate and the magnitude of data correlation.

Conclusions:

• The DBM method is robust for testing modality effects in the null case for typical ROC curves when sufficient sample size is used.
• The findings support the reliability of the DBM method in multireader, multimodality ROC analysis.