Using ratings of resident competence to evaluate NBME examination passing standards

The passing standards of the National Board of Medical Examiners (NBME) examinations were empirically evaluated by analyzing the distributions of examination scores received by 1,994 graduates of one medical school in relation to the clinical competence ratings given to the graduates by their first-year residency directors. A significant association was found between the NBME examination scores and ratings in the cognitive areas of clinical competence. The graduates who scored 420 or less on the NBME Part I or the Part II examination received significantly lower medical knowledge ratings than did the total group of graduates. A similar analysis of NBME Part III examination scores was less clear-cut but also suggested that a score of 420 or less could identify those graduates at significant risk of receiving low knowledge ratings. When low ratings were used as an outcome measure, analysis showed that the NBME Part II examination was not sensitive in detecting such graduates. Based on these data, the authors do not propose changes in the passing standards for the NBME examinations but recommend that these standards continue to be reassessed and further measures be taken to strengthen the internal evaluation methods in medical schools.