Comparison of the performance of visual and verbal multiple-choice questions

A study was carried out to determine whether multiple-choice questions based on visual trigger material, such as micrographs, photographs or radiographs, were more difficult and more discriminating than corresponding verbal questions testing: (i) the same information; (ii) different information in the same content area. Seventy-seven question triplets, each of which contained a visual (X) question, a verbal (Y) question testing the same information and a verbal (Z) question testing different information in the same content area were set in twelve Part 1 FRACS examinations. The questions were arranged in such a way that the X and Z questions in a triplet were set in the same examination, while the Y question was set in the corresponding examination of the previous or the following year. The specific outcomes studied were those on facility index and r biserial discriminating index. An analysis of variance on the data pertaining to each index did not indicate significant differences among the groups. None of the differences between pairs of mean facility indices among question groups X, Y and Z was found to be significant at P less than 0.05, nor were any of the differences between pairs of r biserial discriminating indices found to be significant at P less than 0.05. It was concluded that visual questions set in the Part 1 FRACS examination were not demonstrably more difficult or more discriminating than the corresponding verbal questions. The findings do not detract, however, from the obvious importance of visual recognition for future surgeons.