Evaluation using simulated students for reliability of multiple standardized-patients scoring in clinical performance...

The consistency of rating among different standardized patients (SPs) is very important in clinical performance examinations (CPX).

We evaluated the usefulness of simulated students to assess the reliability of a group of standardized patients in a CPX.

Five SPs in each case were trained to simulate the same patient in a CPX. Ten fourth-year medical student volunteers were selected to act as simulated students. After the simulated students encountered the first SP, they reviewed a video of their performance and sequentially repeated the performance with the other 4 SPs.

The average maximum difference (Deltamax +/- SD) of multiple SPs' percent scores on a simulated student was 18 +/- 7. In 10 pairs of SPs who gave largest Deltamax on a given student, the median Deltamax was 30 (range, 26-38) while median difference of percent score in the faculty scoring on the same simulated student was 3 (range, 0-7). Five problematic SPs whose ratings were significantly different from 3 or more other SPs' ratings in each case were identified.

Simulated students may be a useful means to check the consistency of rating among different SPs in a CPX.