Screening test length for sequential testing with a standardized-patient examination: a receiver operating...

Area of Science:

• Medical Education
• Psychometrics
• Health Professions Education

Background:

• Sequential testing, incorporating a screening test, has proven feasible for reducing the length and cost of performance-based examinations using standardized-patient cases.
• Previous studies demonstrated feasibility, but this research focuses on quantitative criteria for optimizing screening test parameters.

Purpose of the Study:

• To determine optimal quantitative criteria for the length of a screening test and the placement of its pass-fail cutoff within a sequential testing framework.
• To evaluate the psychometric properties of a screening test using receiver operating characteristic (ROC) analysis.

Main Methods:

• Data were collected from five senior medical student classes at Southern Illinois University School of Medicine (1987-1991).
• Receiver operating characteristic (ROC) curves were generated for screening tests of varying lengths.
• Different pass-fail cutoff points on the screening test were analyzed for each ROC curve.

Main Results:

• A screening test approximately one-third the length of the full examination achieved good accuracy.
• Setting the screening pass-fail cutoff slightly above the mean of case pass levels maximized sensitivity and specificity.
• ROC analysis proved valuable in assessing screening test psychometric properties.

Conclusions:

• This study validates the use of ROC analysis for evaluating screening test psychometric properties in sequential testing.
• Optimized sequential testing strategies can enhance the efficiency and accuracy of medical performance examinations.
• Quantitative criteria provide a robust method for determining optimal screening test design.