Comparison of Certainty-Based Marking (CBM) and Number Right Scoring (NRS) in Multiple-Choice Question (MCQ)...

Multiple-choice questions (MCQs) with number right scoring (NRS) are now the most preferred assessment method in medical education, which requires program holders to ensure their fairness for profound decision-making and policy arrangements. Including certainty-based marking (CBM) in MCQ exams is suggested to help better distinguish students and thus increase the validity of assessments. This study aims to integrate CBM into MCQ assessments using a relatively new scoring matrix to assess pass/fail rates and exam scores.

Students in their second year of medical school participated in ten different CBM-MCQ exams. Questions were scored twice: first, traditionally using the NRS, and second, using the CBM (based on correctness and a self-report certainty scale), yielding two different final scores for each student. Exam scores and pass/fail rates were compared using paired t-tests and McNemar tests, respectively.

A total of 935 students in 10 different exams were included in the study. CBM scores were significantly lower than NRS scores (0.82 points; P<0.001). There was a significant shift in the pass/fail classifications (P<0.001). Overall, 34 out of 141 NRS failures (24.1%) passed under CBM, while 85 (10.7%) of the 794 students who passed under NRS failed under CBM. This significant shift was also reported in 5 of 10 exams (P<0.05). Additionally, there was a trend towards worsening scores in CBM compared to NRS. Students with "A" grades decreased from 8.4% to 4.7% while students with "D" grades increased from 15.1% to 20.5%.

CBM yields significantly different scores compared to NRS, as evidenced by distinct pass/fail rates, suggesting the potential for a better assessment tool. Further studies with different types of assessments are needed to validate this method in terms of reducing cheating and guesswork. However, replacing CBM with conventional scoring methods requires further evidence and consideration.