Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Confidence intervals for the receiver operating characteristic area in studies with small samples

N A Obuchowski1, M L Lieber

  • 1Department of Biostatistics and Epidemiology, Cleveland Clinic Foundation, OH 44195-5196, USA.

Academic Radiology
|August 14, 1998
PubMed
Summary

For comparing two receiver operating characteristic (ROC) areas, standard confidence intervals (CIs) work even with small sample sizes. However, for a single ROC area, especially with high accuracy, larger sample sizes are needed for reliable CIs.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Multi-vendor multi-site quantitative MRI analysis of cartilage degeneration 10 Years after anterior cruciate ligament reconstruction: MOON-MRI protocol and preliminary results.

Osteoarthritis and cartilage·2022
Same author

Donor-matched comparison of chondrogenic progenitors resident in human infrapatellar fat pad, synovium, and periosteum - implications for cartilage repair.

Connective tissue research·2019
Same author

Primary Cells Isolated from Human Knee Cartilage Reveal Decreased Prevalence of Progenitor Cells but Comparable Biological Potential During Osteoarthritic Disease Progression.

The Journal of bone and joint surgery. American volume·2018
Same author

Histopathological assessment of primary osteoarthritic knees in large patient cohort reveal the possibility of several potential patterns of osteoarthritis initiation.

Current research in translational medicine·2017
Same author

Meniscus treatment and age associated with narrower radiographic joint space width 2-3 years after ACL reconstruction: data from the MOON onsite cohort.

Osteoarthritis and cartilage·2015
Same author

Comparison of radiation dose and image quality from single-energy and dual-energy CT examinations in the same patients screened for hepatocellular carcinoma.

Clinical radiology·2014

Area of Science:

  • Biostatistics
  • Medical Diagnostics
  • Machine Learning Evaluation

Background:

  • Confidence intervals (CIs) are crucial for assessing the precision of diagnostic test accuracy metrics.
  • Asymptotic methods for CI construction are widely used but may fail under certain sample size conditions.
  • The Receiver Operating Characteristic (ROC) curve and its Area Under the Curve (AUC) are key performance indicators for diagnostic tests.

Purpose of the Study:

  • To determine the minimum sample size required for appropriate asymptotic confidence intervals (CIs) for ROC areas.
  • To identify suitable alternative CI construction methods when sample sizes are below the recommended threshold.

Main Methods:

  • A Monte Carlo simulation study was conducted to evaluate 95% CIs for single and paired ROC areas.

Related Experiment Videos

  • Both parametric and nonparametric estimation methods were used for ROC area calculations.
  • Alternative CI methods, including bootstrap variations and Student t-distribution based CIs, were assessed.

    • Main Results:

    • Asymptotic CIs demonstrated adequate coverage for the difference between two ROC areas, even with small sample sizes (n=20).
    • For a single ROC area, asymptotic methods showed insufficient coverage with small samples.
    • High-accuracy ROC areas required sample sizes exceeding 200 for asymptotic methods to be reliable; bootstrap methods are recommended alternatives.

    Conclusions:

    • The choice of CI method for a single ROC area with small sample sizes is dependent on the estimation approach, data format, and ROC area.
    • No single alternative CI method universally outperforms others for small sample sizes in all scenarios for a single ROC area.