Can family medicine residents predict their performance on the in-training examination?

Evaluation of residents' competence is of great importance in the training of physicians, yet the evaluation process is largely subjective. Faculty and residents frequently rely on self assessments to make decisions regarding curriculum design and electives. The In-training Examination (ITE), the only widely available objective measure of residents' medical knowledge, provides an opportunity to test the reliability of self-assessments. This study's objective was to determine if family medicine residents are able to self-assess their medical knowledge by predicting their performance on the ITE.

A survey asking the residents to estimate their performance on the ITE in each of the nine content areas was administered at 13 examination sites just prior to the ITE. Correlation coefficients were calculated for corresponding predicted and actual scores for each resident in each content area. Predictions were also compared to performance according to quartile.

Residents showed little ability to predict their scores in any of the content areas. Residents scoring in either the lowest or highest quartile were least able to predict accurately, with correct predictions ranging from 3% to 23%.

Residents cannot reliably predict their performance on the ITE. Of special concern are residents scoring in the lowest quartile, since these residents greatly overestimated their performance.