Related Experiment Video
Updated: Feb 3, 2026
Diagnosis of Musculus Gastrocnemius Tightness - Key Factors for the Clinical Examination
Published on: July 7, 2016
Validity Evidence for a Brief Online Key Features Examination in the Internal Medicine Clerkship
Valerie J Lang1, Norman B Berman, Kirk Bronander
1V.J. Lang is associate professor of medicine, director of the medicine subinternship, and senior associate division chief, Hospital Medicine, University of Rochester School of Medicine and Dentistry, Rochester, New York. N.B. Berman is professor of pediatrics and of medical education, Geisel School of Medicine at Dartmouth, Hanover, New Hampshire. K. Bronander is professor of medicine and medical director of simulation, University of Nevada, Reno School of Medicine, Reno, Nevada. H. Harrell is professor of medicine and codirector of the medicine clerkship, University of Florida, Gainesville, Florida. S. Hingle is professor of medicine, director of the Year 3 curriculum, and director of faculty development, Southern Illinois University School of Medicine, Springfield, Illinois. A. Holthouser is professor of medicine and pediatrics and senior associate dean for medical education, University of Louisville, Louisville, Kentucky. D. Leizman is associate professor of medicine and clerkship director for internal medicine, Case Western Reserve University, University Hospital, Cleveland Medical Center, Cleveland, Ohio. C.D. Packer is professor of medicine, Case Western Reserve University, and clerkship director for internal medicine, Louis Stokes Cleveland Veterans Affairs Medical Center, Cleveland, Ohio. Y.S. Park is associate professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois. T.R. Vu is associate professor of clinical medicine, Indiana University School of Medicine, Indianapolis, Indiana. R. Yudkowsky is professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois. S. Monteiro is assistant professor, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada. G. Bordage is professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois.
Purpose:
Medical educators use key features examinations (KFEs) to assess clinical decision making in many countries, but not in U.S. medical schools. The authors developed an online KFE to assess third-year medical students' decision-making abilities during internal medicine (IM) clerkships in the United States. They used Messick's unified validity framework to gather validity evidence regarding response process, internal structure, and relationship to other variables.
Method:
From February 2012 through January 2013, 759 students (at eight U.S. medical schools) had 75 minutes to complete one of four KFE forms during their IM clerkship. They also completed a survey regarding their experiences. The authors performed item analyses and generalizability studies, comparing KFE scores with prior clinical experience and National Board of Medical Examiners Subject Examination (NBME-SE) scores.
Results:
Five hundred fifteen (67.9%) students consented to participate. Across KFE forms, mean scores ranged from 54.6% to 60.3% (standard deviation 8.4-9.6%), and Phi-coefficients ranged from 0.36 to 0.52. Adding five cases to the most reliable form would increase the Phi-coefficient to 0.59. Removing the least discriminating case from the two most reliable forms would increase the alpha coefficient to, respectively, 0.58 and 0.57. The main source of variance came from the interaction of students (nested in schools) and cases. Correlation between KFE and NBME-SE scores ranged from 0.24 to 0.47 (P < .01).
Conclusions:
These results provide strong evidence for response-process and relationship-to-other-variables validity and moderate internal structure validity for using a KFE to complement other assessments in U.S. IM clerkships.
Related Concept Videos
02:55The Evidence for Evolution
01:29Reliability and Validity
02:35Key Elements for Plant Nutrition
02:00Internal Energy
01:29Internal Energy
01:31Internal Receptors