Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

ROC and loss function analysis in sequential testing.

Arno M M Muijtjens1, Scheltus J Van Luijk, Cees P M Van Der Vleuten

  • 1Department of Educational Development and Research, University of Maastricht, P.O. Box 616, 6200, MD, Maastricht, The Netherlands. a.muijtjens@educ.unimaas.nl

Advances in Health Sciences Education : Theory and Practice
|April 4, 2006
PubMed
Summary

Sequential testing in OSCEs reduces costs but risks false positives. This study challenges the application of classic Receiver Operator Characteristic (ROC) analysis, proposing modifications for accurate cutpoint determination in sequential testing.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Online professionalism through the lens of medical students and residents: A focus group study.

International journal of medical informatics·2025
Same author

Computer Adaptive vs. Non-adaptive Medical Progress Testing: Feasibility, Test Performance, and Student Experiences.

Perspectives on medical education·2024
Same author

Composite Reliability of Workplace Based Assessment of International Medical Graduates.

MedEdPublish (2016)·2024
Same author

Studies on Reliability and Measurement Error of Measurements in Medicine - From Design to Statistics Explained for Medical Researchers.

Patient related outcome measures·2023
Same author

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine.

Academic medicine : journal of the Association of American Medical Colleges·2022
Same author

Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity.

Medical teacher·2022

Area of Science:

  • Medical Education
  • Psychometrics

Background:

  • Sequential testing in Objective Structured Clinical Examinations (OSCEs) aims to reduce resource expenditure.
  • This method involves an initial screening test, with candidates passing deemed to have passed the full examination.

Purpose of the Study:

  • To evaluate the validity of applying traditional Receiver Operator Characteristic (ROC) analysis to sequential testing procedures.
  • To investigate the assumptions underlying the equivalence of ROC analysis and loss function minimization in sequential testing.

Main Methods:

  • The study critically examines the theoretical basis for using classic ROC analysis in sequential testing.
  • It contrasts this with a proposed loss function that accounts for false positives and screen negatives.
  • Analysis is performed to assess the conditions under which ROC analysis may be adapted for sequential testing.

Related Experiment Videos

Main Results:

  • The study demonstrates that the assumptions supporting the functional identity of ROC analysis and loss minimization in sequential testing are challengeable.
  • It confirms that ROC analysis can be applied to sequential testing, but requires specific procedural modifications.
  • These modifications are derived from a corresponding loss function analysis.

Conclusions:

  • Traditional ROC analysis is not directly applicable to sequential testing without adjustments.
  • A modified ROC approach, informed by loss function analysis, is necessary for accurate cutpoint selection in sequential OSCEs.
  • This ensures a more valid and cost-effective sequential testing procedure.