Multiple choice question tests in physiology: a preliminary attempt to apply the minimum pass level

At the mid-term test in Part I Physiology at the University of Zagreb the students (n = 280) were graded by our standard pass level (SPL) arbitrarily set at 54% correct answers (SPL = 0.54). The test consisted of 50 items of the one best answer type. Items were selected from the pool by one examiner to conform, by his judgement, to the predetermined SPL. Post hoc the minimum pass level (MPL) was assessed independently by eight examiners and an MPL value of 0.60 for the whole test was obtained. The original Nedelsky scale was used in assessment of MPL but for statistical analysis the data was expressed as log(1/MPL) to linearize the scale of measurements and to reduce the variances. The data showed a large difference between examiners in their assessment of MPL. Nevertheless, the average log(1/MPL) value of individual items showed a significant negative linear relationship with the item difficulty indices as calculated from student's answers, indicating that despite the large heterogenity in assessment the average item log(1/MPL) may be acceptable as a reasonable prediction of item difficulty. Finally, 'subtests' were formed from the whole test by grouping items according to their log(1/MPL) value. The passing rate at these subtests was found to be identical despite the fact that they considerably differed in their MPL values. Therefore, the MPL value seems to be useful in setting objective standards for the decision of pass or fail, even when the MPL was assessed in a very heterogenous way.