A permutation test sensitive to differences in areas for comparing ROC curves from a paired design

Area of Science:

• Biostatistics
• Medical Diagnostics
• Statistical Modeling

Background:

• The area under the ROC curve (AUC) is a key metric for evaluating diagnostic test performance.
• Comparing AUCs is crucial for selecting superior diagnostic systems.
• Existing non-parametric methods for comparing ROC curves have limitations, particularly in paired designs.

Purpose of the Study:

• To develop an exact non-parametric statistical procedure for comparing two ROC curves in paired settings.
• To evaluate the operating characteristics and power of the proposed test through simulations.
• To introduce an asymptotic version for computational efficiency in large sample sizes.

Main Methods:

• Developed an exact non-parametric test based on permutations of subject-specific rank ratings.
• Conducted extensive simulations to assess test performance across various parameters.
• Derived an asymptotic version using an exact estimate of variance in permutation space.

Main Results:

• The proposed exact test demonstrates good operating characteristics and is more powerful than conventional methods for AUC comparisons in paired designs.
• The test is particularly effective for small sample sizes common in diagnostic imaging research.
• The asymptotic version provides a precise and computationally efficient approximation, suitable for large sample sizes.

Conclusions:

• The developed exact non-parametric procedure is a valuable tool for comparing ROC curves in paired designs.
• It offers improved power and reliability, especially in experimental diagnostic imaging studies.
• Both exact and asymptotic versions provide robust statistical comparisons of diagnostic accuracy.