Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

A permutation test for comparing ROC curves in multireader studies a multi-reader ROC, permutation test.

Andriy I Bandos1, Howard E Rockette, David Gur

  • 1Department of Biostatistics, Graduate School of Public Health, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213-3180, USA.

Academic Radiology
|March 24, 2006
PubMed
Summary

A new permutation test effectively compares diagnostic accuracy between two methods in multi-reader studies. This statistical approach enhances the analysis of receiver operating characteristic (ROC) curves, aiding in the selection of superior diagnostic tools.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Training AI to Improve Distinction of Triple-Negative Invasive Breast Cancer from Cysts and Fibroadenomas on Ultrasound.

Diagnostics (Basel, Switzerland)·2026
Same author

Evaluation of In Vitro Efficiency of Ciclopirox Against <i>Yersinia pestis</i> and <i>Francisella tularensis</i>.

International journal of molecular sciences·2026
Same author

Pfirrmann Grading and Features of Internal Derangement Identified in Discs Studied by Lumbar Discography in Patients with Chronic Low Back Pain.

AJNR. American journal of neuroradiology·2025
Same author

Screening for Breast Cancer with Contrast-enhanced Mammography as an Alternative to MRI: SCEMAM Trial Results.

Radiology·2025
Same author

STOPPING RULES FOR LONG TERM CLINICAL TRIALS BASED ON TWO CONSECUTIVE: REJECTIONS OF THE NULL HYPOTHESIS.

Communications in statistics: theory and methods·2025
Same author

Novel Bivalent mRNA-LNP Vaccine for Highly Effective Protection against Pneumonic Plague.

Advanced science (Weinheim, Baden-Wurttemberg, Germany)·2025

Area of Science:

  • Biostatistics
  • Medical Diagnostics
  • Statistical Modeling

Background:

  • Comparing diagnostic accuracy is crucial for selecting optimal medical tests.
  • Existing methods may not adequately address multireader paired study designs.
  • Receiver operating characteristic (ROC) curve analysis is a standard tool for evaluating diagnostic performance.

Purpose of the Study:

  • To develop a novel permutation test for comparing ROC curves from two diagnostic modalities.
  • To specifically address a multireader paired design where reader variability is accounted for.
  • To provide a statistically rigorous method for detecting differences in diagnostic accuracy.

Main Methods:

  • A permutation test was developed based on all possible exchanges of reader ratings between modalities.

Related Experiment Videos

  • An exact permutation test was constructed using extreme values of the average difference in areas under the ROC curves (AUCs).
  • An asymptotic version of the test was derived, and computer simulations validated its type I error rate for small sample sizes.

    • Main Results:

    • The developed permutation test is suitable for multireader paired designs with fixed reader effects.
    • The asymptotic test demonstrated accurate type I error rates even with small sample sizes (e.g., 20 normal and 20 abnormal cases).
    • The test is specifically designed to detect differences in AUCs between the two diagnostic modalities.

    Conclusions:

    • The proposed permutation test offers a powerful statistical method for comparing diagnostic modalities.
    • This approach is particularly valuable in multireader paired study designs.
    • The primary utility lies in detecting differences in average AUCs, aiding in the selection of the more accurate diagnostic tool.