Computerization of a patient management problems examination to prevent 'retracing'

Area of Science:

• Medical Education
• Assessment and Evaluation
• Pediatric Training

Background:

• The Pediatric Medicine Examination (PMP) is a crucial assessment tool for pediatricians.
• Understanding the impact of test format on examinee performance is essential for valid assessment.
• The phenomenon of 'retracing' in written tests can influence results.

Purpose of the Study:

• To compare the performance of pediatric trainees and candidates on written versus computerized PMP formats.
• To investigate the effect of computerization on examinee errors and diagnostic accuracy.
• To evaluate the impact of computerization on test reliability and validity.

Main Methods:

• A 10-patient, linear PMP was administered in written and computerized formats.
• 104 candidates for American Board of Pediatrics certification and 122 first-month pediatric residents participated.
• Statistical analyses compared error rates, diagnostic accuracy, and correlations with other examinations.

Main Results:

• Examinees made more omission errors and fewer correct diagnoses on the computerized PMP.
• Computerization appeared to prevent 'retracing,' a phenomenon observed in the written format.
• Test reliability, distinction between trainees and candidates, and correlations with oral exams were similar for both formats.

Conclusions:

• Computerizing the PMP controls for the extraneous variable of 'retracing' without compromising reliability or validity.
• The computerized PMP offers a potentially more accurate measure of examinee knowledge and decision-making.
• Assessment format significantly influences examinee behavior and performance metrics in medical examinations.