Students' certainty during course test-taking and performance on clerkships and board exams

Psychometric aspects of multiple-choice tests were investigated using a confidence-weighted scoring technique. The contributions of two indices, overconfidence and underconfidence, in the prediction of subsequent academic performance of examinees were studied. A total of 444 sophomore students (entering classes of 1982 and 1983) in one medical school were asked to indicate their confidence, on a 5-point scale (100, 75, 50, 25, and 0), in the correctness of their responses to each multiple-choice item on an Introduction to Clinical Medicine examination. Examinations were scored in two ways: in the conventional way, using the total number of correct responses, and by a confidence-weighted technique based on the level of certainty indicated for each response by the examinee. Only the conventional score determined the grade; the confidence-weighted score was calculated for the purely experimental purposes of this study. Overconfidence and underconfidence indices were also calculated by using the indicated levels of certainty. Improvements in the psychometrics of the examinations were observed when confidence-weighted scoring was used. In multiple-regression models, the confidence-weighted scores and indices of over- and underconfidence contributed significantly to predicting scores of the students studied on Parts I and II of the National Board of Medical Examiners examinations, whereas the conventional score did not contribute to the prediction of Part II scores. Significant differences on junior clerkship examinations and ratings were observed between those who were highly overconfident and those who were slightly overconfident. The highly overconfident students also estimated higher future incomes than did those who were slightly overconfident.