Measuring the problem-solving ability of students and residents by microcomputer

Potential uses for microcomputers in medical education include evaluation of students and residents. A computer program developed at the University of Pennsylvania School of Medicine and the Children's Hospital of Philadelphia presents simulated patient cases and then scores participants' clinical problem-solving in the cases by comparing their performances with those of five faculty members. In the study reported here, the authors investigated the validity and reliability of this evaluation system. Total scores were significantly different among faculty members, residents, and third-year students on only the most difficult of the three cases included in the study. Performance was not related to the user's general analytic ability or to locus of control. This result suggested there is not a "type" of person who is better at medical problem-solving on a microcomputer. A prophecy formula was used to determine that nine cases would be needed to achieve an adequate level of reliability in measuring the ability of a student or a resident. Implications of selecting a battery of cases for the system are discussed.