Use of NBME and USMLE examinations to evaluate medical education programs

Criteria are presented for determining whether licensure and/or achievement test results should be used as a basis for making various types of comparisons and judgments about medical education programs for program evaluation purposes. United States Medical Licensing Examination (USMLE) examinations and National Board of Medical Examiners (NBME) subject tests are then considered as sources of evaluation data for medical education programs by evaluating these examinations in the light of the proposed criteria. It is concluded that NBME examination and USMLE results may properly be considered one of several tools to use in an evaluation of medical education programs but are not recommended as the sole basis for a judgment of program effectiveness. However, care must be taken to ensure that test score differences are large enough to be real and to have practical significance. Furthermore, changes in curriculum should not be considered or implemented on the basis of NBME/USMLE results unless the changes in test scores are maintained for at least two years. The focus should generally be on inordinately high failure rates rather than on changes in mean class scores. Efforts to develop objective, standardized, and controlled measures of other clinical competencies for national use should be encouraged to minimize the temptation to use USMLE Step 1 and Step 2 scores as the sole basis for comparisons of medical education programs.