Evaluating and improving multiple choice papers: true-false questions in public health medicine

The quality of a multiple true-false (MTF) examination paper in public health medicine for 149 clinical medical students was evaluated using predefined performance criteria to offer guidelines for improvement of such a paper. There were 35 questions, each with five true-false branches, and the performance of the overall best 25% of candidates was compared for individual items with that of the overall worst 25%. To improve discrimination between best and worst candidates, 60% of items needed changes, and several indicators were used to identify how, usually because the branch was too easy (26%), unpopular (16%) or too hard (10%). A number of guidelines for writing good MTF questions and for improving them are suggested. The inequity is illustrated of marking systems which do not allocate a negative mark for incorrect answers equal in size to the positive mark for correct ones, with zero for unanswered questions or 'don't know' answers.