Comparability of decisions for computer adaptive and written examinations

The purpose of this study was to determine the comparability of examinee ability measures, pass/fail decisions, and confidence in the accuracy of pass/fail decisions on written fixed-length and computer adaptive examinations. Medical technology students took two independent tests: a written test of 109 questions and a computer-administered computer adaptive test that included 50 to 100 items tailored to the ability of each student. Results indicated that ability measures on the two tests correlated at 0.84. Decisions were made with 90% confidence in their accuracy for 72% of the examinees on the computer adaptive test, and for 58% of the examinees on the traditional written test. The decision accuracy on the computer adaptive test was higher even though the test was shorter because the tailoring process reduced the error of measure.