Confidence weighted answer technique in a group of pediatric residents

Physicians who are excessively underconfident or overconfident about their knowledge may have impaired clinical judgement. Confidence weighting of multiple choice examinations asks test-takers to state how confident they are that the answers they selected are correct. This previously described method allows the examinee to receive 'overconfidence' and 'underconfidence' scores. This study was designed to test the hypothesis that these scores would correlate with faculty assessment of pediatric residents' confidence level as observed in the clinical setting. Thirty-three pediatric residents took an examination of general pediatric knowledge using confidence weighting method. Percentage of questions answered correctly ranged from 40 to 81%. There was an association between increasing overconfidence and lower examination scores (r = 0.58; p = 0.001). Increasing overconfidence was also associated with decreasing underconfidence (r = 0.38; p = 0.04). Five faculty members, the program director and the chief resident were asked to rate their perceptions of the residents' confidence on a Likert-type scale. The period of observation ranged from 9 months to 3 years. Linear regression demonstrated an association between underconfidence indices and observed confidence in the clinical setting (r = 0.39; p = 0.03). In addition, three of four residents who left the program had either over- or underconfidence indices greater than one standard deviation from the mean. These results indicate that the multiple choice examination with confidence weighting can predict residents who will be judged as underconfident by clinical preceptors. This finding is important in light of our impression that such house officers often have difficulties later during their residencies.