Effects of using two or more standardized patients to simulate the same case on case means and case failure rates

The effects of using two or more standardized patients (multiple SPs) to simulate the same case in a performance-based examination were studied at the case level by comparing case means and case failure rates for multiple SPs simulating the same case, using data from the classes of 1988, 1989, and 1990 at the Southern Illinois University School of Medicine. For total scores and scores on the students' written answers, the effects on means and failure rates were negligible and could be explained as due to sampling error. For scores on the checklists completed by the SPs, there were more significant differences than would be expected by chance alone, even though the number of significant differences was relatively small. The results demonstrate a need for caution in the interpretation of scores obtained from a case checklist completed by multiple SPs, particularly in regard to making pass-fail decisions.