Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Assessing classifiers from two independent data sets using ROC analysis: a nonparametric approach.

Waleed A Yousef1, Robert F Wagner, Murray H Loew

  • 1Food and Drug Administration, Center for Devices and Radiological Health, Rockville, MD 20852, USA. wyousef@aucegypt.edu

IEEE Transactions on Pattern Analysis and Machine Intelligence
|October 27, 2006
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Is it blur or is it texture? An analysis of computational blur and texture measures commonly used in medical image analysis.

Journal of medical imaging (Bellingham, Wash.)·2023
Same author

Brief history of Image Processing at SPIE Medical Imaging.

Journal of medical imaging (Bellingham, Wash.)·2022
Same author

Unmasking a Privacy Concern: Potential Identification of Patients in an Immobilization Mask from 3-Dimensional Reconstructions of Simulation Computed Tomography.

Practical radiation oncology·2021
Same author

Adverse events of after-loading high dose rate brachytherapy reported to the United States Food and Drug Administration (FDA).

Brachytherapy·2021
Same author

Threshold for Tonic Motor Effects from Random Waveform in a Rat Experimental Model of Frontal Cortex Stimulation.

Stereotactic and functional neurosurgery·2020
Same author

Heat transfer analysis and resolution quantification of active dynamic thermography through human skin.

Lasers in surgery and medicine·2018
Same journal

Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective.

IEEE transactions on pattern analysis and machine intelligence·2026
Same journal

Learning Topology-Aware Representations via Test-Time Adaptation for Anomaly Segmentation.

IEEE transactions on pattern analysis and machine intelligence·2026
Same journal

TraGraph-GS: Trajectory Graph-based Gaussian Splatting for Arbitrary Large-Scale Scene Rendering.

IEEE transactions on pattern analysis and machine intelligence·2026
Same journal

SWIFT: A Small-World Interaction Framework for Flow-Aware Trajectory Prediction in Autonomous Driving.

IEEE transactions on pattern analysis and machine intelligence·2026
Same journal

HardFlow: Hard-Constrained Sampling for Flow-Matching Models Via Trajectory Optimization.

IEEE transactions on pattern analysis and machine intelligence·2026
Same journal

Industrial Brain: Self-Evolving Neuro-Symbolic Autonomy with Causal Resilience for Cyber-Physical Systems.

IEEE transactions on pattern analysis and machine intelligence·2026
See all related articles

This study introduces a nonparametric method to estimate the Area Under the ROC Curve (AUC) and its variance for binary classification. The approach provides insights into the sources of uncertainty in AUC estimation, particularly with limited data.

Area of Science:

  • Machine Learning
  • Statistical Modeling
  • Data Science

Background:

  • Binary classification performance is often evaluated using the Area Under the ROC Curve (AUC).
  • Estimating the uncertainty associated with AUC is crucial for reliable model assessment.
  • Existing methods may require distributional assumptions or large datasets.

Purpose of the Study:

  • To develop a nonparametric method for estimating conditional AUC and its variance.
  • To derive a closed-form expression for the variance of the AUC estimator.
  • To provide a framework for understanding sources of uncertainty in AUC estimation.

Main Methods:

  • Utilized U-statistics for nonparametric estimation.
  • Derived a closed-form expression for the variance of the AUC estimator.

Related Experiment Videos

  • Applied methods to binary classification tasks with distinct training and testing sets.

    • Main Results:

    • Successfully estimated conditional AUC, its mean, and variance.
    • Derived a novel closed-form expression for AUC estimator variance.
    • Identified key components contributing to the uncertainty in AUC estimates.
    • Demonstrated the utility of the estimators through simulation results.

    Conclusions:

    • The proposed nonparametric U-statistics approach offers a robust way to assess AUC and its uncertainty.
    • The derived variance expression enhances understanding of AUC estimation reliability.
    • This method is valuable for binary classification when data distributions are unknown.