Computerization of a patient management problems examination to prevent 'retracing'

During 1973, a 10-patient, linear PMP examination was administered in both written and computerized form to 104 candidates for certification by the American Board of Pediatrics and to 122 individuals in their first month of graduate training in paediatrics. Both groups made significantly more errors of omission (failure to select appropriate actions) and fewer correct diagnoses on the computerized version of the test than on the written version. These results suggest that examinees 'retrace' (use clues from problems near the end of the case to make additional correct decisions on earlier problems) when the test is presented in written form and that computerization prevents this phenomenon. Additional analyses indicated that the reliability of the PMP examination was essentially the same for the written and computerized versions, that both forms distinguished about equally well between first-year residents and candidates, and that both forms had about the same magnitude of correlation with the American Board of Pediatrics oral examination. Finally, correlation between both PMP forms and the American Board of Pediatrics multiple-choice examination suggested that the PMP, in either mode of presentation, measures characteristics that are not tapped by the multiple-choice test. Thus, computerization of the PMP provides a means for controlling a variable (retracing) which may be considered extraneous to the desired measurement, but it does not appear to alter the reliability of the test or to modify its relations with other measures.