Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

"Binary" and "non-binary" detection tasks: are current performance measures optimal?

David Gur1, Howard E Rockette, Andriy I Bandos

  • 1Department of Radiology, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA. gurd@upmc.edu

Academic Radiology
|July 14, 2007
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Training AI to Improve Distinction of Triple-Negative Invasive Breast Cancer from Cysts and Fibroadenomas on Ultrasound.

Diagnostics (Basel, Switzerland)·2026
Same author

Evaluation of In Vitro Efficiency of Ciclopirox Against <i>Yersinia pestis</i> and <i>Francisella tularensis</i>.

International journal of molecular sciences·2026
Same author

Pfirrmann Grading and Features of Internal Derangement Identified in Discs Studied by Lumbar Discography in Patients with Chronic Low Back Pain.

AJNR. American journal of neuroradiology·2025
Same author

Screening for Breast Cancer with Contrast-enhanced Mammography as an Alternative to MRI: SCEMAM Trial Results.

Radiology·2025
Same author

STOPPING RULES FOR LONG TERM CLINICAL TRIALS BASED ON TWO CONSECUTIVE: REJECTIONS OF THE NULL HYPOTHESIS.

Communications in statistics: theory and methods·2025
Same author

Novel Bivalent mRNA-LNP Vaccine for Highly Effective Protection against Pneumonic Plague.

Advanced science (Weinheim, Baden-Wurttemberg, Germany)·2025
Same journal

MRI-based Predictors and Risk Constellations of Chronic Ankle Instability After Acute Lateral Ankle Sprain: A Multicenter Study.

Academic radiology·2026
Same journal

Early Prediction of Pathological Complete Response to Neoadjuvant Chemotherapy in Breast Cancer using a Longitudinal US-based Stack-model.

Academic radiology·2026
Same journal

Evaluating the Impact of Embolization on Outcomes in Iliopsoas Hematomas: A Multicenter Retrospective Propensity-matched Study.

Academic radiology·2026
Same journal

Comparison of Iterative Metal Artifact Reduction Presets In Ultra-high-resolution Photon-counting CT Angiography of Patients with Total Knee Endoprosthesis.

Academic radiology·2026
Same journal

Deep Learning for Opportunistic Vertebral Fracture Detection on Routine Thoraco-abdominal Computed Tomography: A Systematic Review and Hierarchical Summary Receiver Operating Characteristic Meta-analysis of Patient-level Diagnostic Test Accuracy.

Academic radiology·2026
Same journal

"Where are They Now?": A Single Institution's 10-Years Experience with an Integrated Nuclear Radiology Fellowship.

Academic radiology·2026
See all related articles

Observer studies often show extreme responses, questioning rating scale validity. Binary ratings may offer a more precise and powerful alternative for detection tasks.

Area of Science:

  • Medical imaging analysis
  • Observer performance studies

Background:

  • Multicategory rating scales are frequently used in observer studies for medical image detection tasks.
  • A significant proportion of responses in these studies tend to fall at the extreme ends of the rating scale (e.g., <11% or >89%).

Purpose of the Study:

  • To investigate the validity and appropriateness of multicategory rating scales in observer detection tasks.
  • To compare the performance of binary versus multicategory rating scales using Monte Carlo simulations.

Main Methods:

  • Analysis of response data from observer studies for detection tasks.
  • Monte Carlo simulations were employed to model both binary and multicategory rating processes.
  • Comparison of summary indices derived from both rating methods.

Related Experiment Videos

Main Results:

  • A large fraction of observer responses in detection tasks cluster at the extreme ends of multicategory scales, irrespective of abnormality presence or subtlety.
  • Monte Carlo simulations indicated that binary rating scales often yield less biased and more precise summary indices compared to multicategory scales.
  • Binary ratings demonstrated potential for higher statistical power in detecting differences between imaging modalities.

Conclusions:

  • The tendency for extreme responses raises concerns about the reliability of multicategory rating scales in observer studies.
  • Binary rating scales may be a more appropriate and statistically powerful method for certain detection tasks in observer performance research.
  • Further investigation into rating scale methodology is warranted to improve the validity and precision of observer study findings.