Criterion-based open book examinations in physiology

Analysed data from ten consecutive physiology examinations for medical students at Aarhus University are presented. Each examination comprised one MCQ test and one short-answer test. For both written aids were allowed; both were dominated by items above the recall level. The mean test scores of both the MCQ and the short answer tests were slightly above their mean pass levels with coefficients of variation around seven. Large but similar pass rate fluctuations in both test were found, and about half of the non-passing students failed both tests. The MCQ minimum pass level, established by the Nedelsky principle, was more stable than the essay pass level. The Nedelsky values seem to compensate for a large part of the fluctuations in the difficulty level of various MCQ sets. Due to lack of comparable data it cannot be settled whether the pass rate fluctuations are excessive, nor whether standards fluctuate more than those at other institutes.