Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science...

Area of Science:

• Educational Assessment
• Cognitive Science

Background:

• Higher education aims to foster problem-solving and higher-order learning.
• Multiple-choice questions (MCQs) are debated for their ability to assess higher-order learning versus rote memorization.
• Existing frameworks for defining higher-order learning and writing MCQs are subjective and criticized.

Purpose of the Study:

• To test guidance for writing MCQs that assess higher-order learning.
• To empirically validate MCQs as a tool for measuring complex cognitive skills.

Main Methods:

• Developed and applied specific guidance for crafting MCQs targeting higher-order cognitive skills.
• Compared the performance of subject matter novices and experts on these MCQs.
• Assessed the ease of answering lower-order questions through online searches.

Main Results:

• MCQs written using the new guidance were significantly more challenging for novices compared to experts.
• Lower-order MCQs were easily answerable by novices through simple online searches.
• The findings provide empirical support for the effectiveness of the developed guidance.

Conclusions:

• The developed guidance effectively creates MCQs that measure higher-order learning.
• These MCQs offer a reliable and potentially cheating-resistant alternative to traditional assessments like essays.
• MCQs can be a valid tool for evaluating complex cognitive abilities in higher education.