Evaluation of vignette-type examination items for testing medical physiology

Step 1 of the United States Medical Licensing Examination has now replaced the National Board of Medical Examiners part 1 examination as the initial step in medical licensure. The new examination format places an increased emphasis on clinical vignettes to test basic physiological concepts. Suggestions from a question-writing workshop were incorporated into cardiovascular items in a midcourse physiology examination. Item analysis indicated that the overall examination difficulty was not different from examinations administered in the preceding years. Within the examination, however, the vignette questions had significantly higher discrimination indexes than other question types. The slope of the difficulty-to-discrimination relationship was significantly higher in 1992 than in previous years. This means that questions of equivalent difficulty were more discriminating in 1992 than in previous years. Finally, this report suggests a method of identifying questions with both high discrimination and high difficulty index for the development of an (informal) test item bank. The clinical vignette question types are a valuable method of examining basic physiological principles.