A comparison of short and multiple choice questions in the evaluation of students of biochemistry

Premedical and other undergraduate students were given an examination consisting of a section of short questions and a section of multiple choice questions of the one in five type. After correction for random guessing, the average multiple choice mark was close to the average short question mark. Thus the theoretical 'average' student did equally well in both types of examination. Individual multiple choice marks were plotted against the corresponding short question mark. Although the two sets of marks were highly correlated (P less than 0.001), there was still a wide scatter around the best-fit, least-square line (r = 0.63). Thus dependence on one type of examination alone could have seriously penalized some students. The least-squares line was extrapolated to determine the multiple-choice mark of a theoretical 'idiot' student, defined as one obtaining zero marks in the short question section of the examination. This student did better than predicted on the basis of random guessing. On the other hand a theoretical 'genius' student, defined as one obtaining top marks in the short question section, did less well than predicted. Thus multiple choice examinations, even when corrected for the random guessing component, may overestimate weaker students and underestimate stronger students.