Measuring the instructional validity of clinical simulation problems

Written clinical simulation problems in two formats--forced-choiced and essay--were used to test junior and senior medical students at the conclusion of their pediatric rotations. A comparison was made in the performance of students with varying levels of clinical experience. There seemed to be no consistent pattern of improvement with increased instructional time on the forced-choice management problems. Junior students did show improvement over time on the essay management problems, which also seemed to reflect the increased problem-solving and organizational skills of seniors in comparison to juniors. Correlations across problems and correlations between the clinical problem test and other measurement techniques were very weak, partly due to the low sampling reliability of clinical simulation problems. In this study, clinical simulation problems failed to demonstrate responsiveness to development and maturation in the problem-solving approach to patient care.