Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Multiple classification and receiver operating characteristic (ROC) analysis.

W R Steinbach1, K Richter

  • 1Academy of Science of the GDR, Department of Radiology, Berlin.

Medical Decision Making : an International Journal of the Society for Medical Decision Making
|October 1, 1987
PubMed
Summary

The receiver operating characteristic (ROC) curve accurately measures observer performance in medical image analysis. This method validates classification accuracy for identifying cardiovascular conditions in chest X-rays.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Data assimilation in the decision support system rodos.

Radiation protection dosimetry·2003
Same author

Increasing sample throughput in pharmacological studies by using dual-column liquid chromatography with tandem mass spectrometry.

Journal of chromatography. A·2003
Same author

Neurotropic effect of juvenile hormone III in larvae of the cockroach, Periplaneta americana.

Journal of insect physiology·2003
Same author

Reproducibility of forced inspiratory and expiratory volumes after bronchodilation in patients with COPD or asthma.

Respiratory medicine·2003
Same author

The effect of storage on allergen and microbial agent levels in frozen house dust.

Allergy·2003
Same author

Dose reduction of inhaled corticosteroids under concomitant medication with montelukast in patients with asthma.

The European respiratory journal·2002

Area of Science:

  • Medical Imaging Analysis
  • Radiology
  • Diagnostic Performance Evaluation

Background:

  • Observer performance is crucial in medical diagnosis, particularly in interpreting complex images like photofluorograms.
  • Traditional performance metrics may not fully capture the nuances of diagnostic decision-making under uncertainty.
  • Receiver Operating Characteristic (ROC) analysis offers a robust framework for evaluating diagnostic accuracy.

Purpose of the Study:

  • To apply Receiver Operating Characteristic (ROC) curve analysis to observer performance in a multiple-alternative decision task.
  • To demonstrate that the area under the ROC curve is a valid performance criterion equivalent to the probability of correct classification.
  • To validate the ROC method using a real-world dataset of chest X-rays for cardiovascular condition detection.

Related Experiment Videos

Main Methods:

  • Utilized degrees of confidence (0-10) for both classification and ROC rating in observer judgments.
  • Applied ROC curve analysis to the classification data from experienced radiologists examining 1,190 photofluorograms.
  • Analyzed classification matrices for radiologists with high, medium, and low performance ratings.

Main Results:

  • The area under the ROC curve was shown to correspond to the probability of correct classification.
  • ROC curves generated were symmetric with points generally located around the off-diagonal.
  • Minimal differences (0.004-0.011) were observed between the overall probability of correct classification and the ROC curve index for individual evaluators.

Conclusions:

  • ROC curve analysis provides a valid and reliable method for assessing observer performance in diagnostic tasks involving medical imaging.
  • The ROC method effectively quantifies diagnostic accuracy, correlating well with traditional classification probabilities.
  • This approach is applicable to evaluating radiologist performance in identifying cardiovascular conditions from chest X-rays.