Effects of publishing multiple-choice questions on their subsequent item analysis

A prospective study was carried out to determine what effects publication within a multiple-choice question (MCQ) book of 63 selected items from a bank had on their performance in subsequent examinations. A closely corresponding group of 63 unpublished questions set both before and after publication of the book was used as control. The specific outcomes studied were those on facility index, r biserial discriminating index and differences in performance between abler (upper 25%) and weaker (lower 25%) candidates. The mean facility index of the published questions increased significantly (P less than 0.01) after publication, while no significant change was observed in the mean facility index of control questions (P greater than 0.05) or in the mean discriminating index of published (P greater than 0.05) and control (P greater than 0.05) questions. Contrary to expectations the mean facility index of published questions increased, after publication, to a greater extent (P less than 0.01) for abler candidates than for weaker candidates. The study supports the hypothesis that publication of questions from a bank makes them easier to answer. The practice of adjusting the cut-off mark for passing according to the proportion of published questions in a given examination, however, is not justified. The findings highlight the need to advise less able candidates to make greater use of the MCQ book in preparing for an examination.