The use of an objective structured clinical examination with postgraduate residents in pediatrics

To evaluate the usefulness of an objective structured clinical examination (OSCE) for assessing and providing feedback to postgraduate residents in pediatrics.

A 5-station OSCE given in 1996, based on the educational objectives of a general pediatric training program. Each station assessed the residents' interviewing and history-taking skills with a standardized patient. The results were correlated with those of the in-training evaluation reports.

The Department of Paediatrics, University of Toronto Faculty of Medicine, Toronto, Ontario.

Forty-three of 61 pediatric residents: 14 first-year, 12 second-year, 8 third-year, and 9 fourth-year residents.

Scores for each of the 5 stations were broken down into 15 points for the check-list, 5 for the global assessment, and 10 for the written postencounter question, for a total score of 150. The in-training evaluation report ratings were converted to a 5-point numerical scale, ranging from 1 (unsatisfactory) to 5 (outstanding).

The mean OSCE score for the 43 pediatric residents was 104.9. Although the residents in their senior year scored higher, there was no statistically significant difference among the 4 years for the total OSCE score or for any of the 5 stations. The fourth-year residents' scores on the postencounter questions were significantly (P < .05) higher than the first-year residents' scores. Two residents scored less than 60%. The internal consistency of the 5-station OSCE was limited (r = 0.69). Residents received verbal feedback at the conclusion of the OSCE, and they received their scores when they were calculated. The mean overall in-training evaluation report score for all 61 pediatric residents was 3.9. There was a moderate, but statistically significant, correlation between the overall mean OSCE results and the overall mean in-training evaluation report scores (r = 0.45).

The OSCE can provide a useful formative evaluation of postgraduate residents, but the usefulness of the evaluation data and the feedback must be balanced with the logistic difficulties and expense.