The average receiver operating characteristic curve in multireader multicase imaging studies.

Area of Science:

• Medical Imaging
• Biostatistics
• Diagnostic Test Evaluation

Background:

• Area under the ROC curve (AUC) is a common metric in multireader, multicase (MRMC) studies for medical imaging.
• Limitations of AUC necessitate complementary visualization methods like average ROC curves.
• Current methods for averaging ROC curves may not accurately reflect the average AUC.

Purpose of the Study:

• To investigate and present methods for generating average ROC curves from individual reader ROC curves in MRMC studies.
• To ensure the generated average ROC curve preserves the area under the curve (AUC) property.
• To evaluate the effectiveness of proposed methods using simulated and real-world data.

Main Methods:

• Developed both non-parametric and parametric methods for averaging ROC curves.
• Ensured the proposed methods are "area preserving," meaning the AUC of the average ROC curve equals the average of individual AUCs.
• Utilized hypothetical, simulated, and real-world medical imaging datasets for validation.

Main Results:

• The proposed non-parametric and parametric methods were demonstrated to be area preserving.
• Averaging ROC curve parameters (conventional or proper bi-normal) was shown to be generally not area preserving.
• Parameter averaging can lead to average ROC curves that do not intuitively represent the mean of individual curves.

Conclusions:

• The developed area-preserving methods are valuable for visualizing average ROC curves in MRMC studies.
• These average ROC curves serve as effective companions to statistical inference on AUC.
• Freely available software implementing these methods supports their practical application in medical imaging research.