Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Degeneracy and discrete receiver operating characteristic rating data

D D Dorfman1, K S Berbaum

  • 1Department of Psychology, University of Iowa, Iowa City 52242, USA.

Academic Radiology
|October 1, 1995
PubMed
Summary

A new algorithm, RSCORE4, effectively handles degenerate data sets in receiver operating characteristic (ROC) curve analysis. It provides more accurate diagnostic accuracy estimates than existing methods like ROCFIT and the Wilcoxon statistic.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Multiple diagnostic task performance in CT examination of the chest.

The British journal of radiology·2012
Same author

Multiple diagnostic task performance in CT examination of the chest.

The British journal of radiology·2012
Same author

Satisfaction of search in osteoradiology.

AJR. American journal of roentgenology·2001
Same author

Factors influencing visualization of vertebral metastases on MR imaging versus bone scintigraphy.

AJR. American journal of roentgenology·2001
Same author

Gaze dwell times on acute trauma injuries missed because of satisfaction of search.

Academic radiology·2001
Same author

Correcting errors in eye-position data arising from the distortion of magnetic fields by display devices.

Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc·2001

Area of Science:

  • Medical Imaging
  • Biostatistics
  • Diagnostic Accuracy Studies

Background:

  • Observer performance studies require accurate diagnostic accuracy estimation.
  • Degenerate data sets pose a significant challenge in receiver operating characteristic (ROC) curve analysis.
  • Existing methods like ROCFIT and Wilcoxon statistic have limitations in handling degenerate data.

Purpose of the Study:

  • To introduce and evaluate a new algorithm, RSCORE4, for addressing degenerate data sets in ROC analysis.
  • To compare the performance of RSCORE4 against ROCFIT and the Wilcoxon statistic.
  • To improve the estimation of diagnostic accuracy in observer performance studies.

Main Methods:

  • RSCORE4 utilizes a maximum-likelihood approach with a small constant added to empty cells to resolve degeneracy.

Related Experiment Videos

  • When the initial method fails, RSCORE4 employs a pattern-search algorithm.
  • Performance was evaluated through Monte Carlo simulations.

    • Main Results:

    • RSCORE4 consistently converged to non-degenerate solutions across all simulations.
    • RSCORE4 yielded results closer to population values compared to ROCFIT and Wilcoxon.
    • ROCFIT produced degenerate solutions in over 40% of samples, and the Wilcoxon statistic showed bias.

    Conclusions:

    • RSCORE4 demonstrates superior performance in handling degenerate data sets for ROC analysis.
    • The algorithm offers a more reliable method for estimating diagnostic accuracy.
    • RSCORE4 is recommended as an improvement over current methods for dealing with degeneracy in ROC studies.