Certainty and safe consequence responses provide additional information from multiple choice question assessments

Area of Science:

• Medical Education
• Cognitive Psychology
• Clinical Decision-Making

Background:

• Effective clinical decision-making requires self-monitoring and awareness of action consequences.
• Assessing medical students' self-monitoring abilities can be achieved through multiple-choice questions (MCQ) by evaluating response certainty and avoidance of unsafe options.

Purpose of the Study:

• To investigate the relationship between medical students' response certainty, the safety of their incorrect answers, and overall performance in a summative MCQ examination.
• To determine if response certainty and consequence provide additional predictive information beyond traditional MCQ scoring.

Main Methods:

• Fifth-year medical students (n=330) indicated their certainty for each MCQ response during a summative exam.
• Incorrect responses were evaluated by an expert panel for their inherent level of unsafeness (response consequence).
• Analyses compared response certainty and consequence across different student performance levels.

Main Results:

• Increased certainty in responses correlated with higher odds of being correct and lower odds of providing unsafe answers.
• For some student ability groups, high certainty on incorrect responses increased the likelihood of those responses being unsafe.
• Students below the standard performance level demonstrated appropriate certainty.

Conclusions:

• Response certainty and the consequence of incorrect answers offer valuable information beyond the number of correct responses in MCQs.
• Measures of certainty and consequence are relatively independent of traditional scoring and can be particularly useful for students near the pass-fail threshold.
• While appropriate certainty was observed, high certainty on incorrect answers, except for the lowest performers, indicated a greater risk of unsafe choices.