The cognitive structure of the modified essay question

Area of Science:

• Medical Education
• Cognitive Assessment
• Psychometrics

Background:

• The Final MB Part II examination at Queen's University, Belfast, utilizes modified essay questions (MEQs).
• Cognitive abilities are often categorized using frameworks like Bloom's taxonomy and Buckwalter's cognitive levels.
• Assessing higher-order cognitive skills in medical examinations is crucial for effective clinical practice.

Purpose of the Study:

• To analyze the extent to which MEQs in the Final MB Part II examination assess Buckwalter's three levels of cognitive ability.
• To compare the cognitive level distribution between the 1978 and 1980 MEQ papers.
• To investigate the relationship between MEQ scores and performance in the Final Clinical examination.

Main Methods:

• Analysis of 1978 and 1980 Final MB MEQs question by question.
• Application of Bloom's taxonomy and Buckwalter's cognitive levels for assessment.
• Factor analysis of scores to determine underlying relationships between questions.

Main Results:

• MEQs demonstrated an uneven distribution of cognitive levels (Buckwalter's Levels I, II, III) within and between examination years.
• A strong correlation was found between MEQ scores and Final Clinical examination scores.
• Factor analysis indicated that comprehension and evaluation questions did not dominate any single factor.

Conclusions:

• The 1980 MEQ paper primarily tested 'knowledge,' while the 1978 paper assessed higher-order skills like analysis, synthesis, and evaluation.
• The design of MEQs needs to ensure an appropriate balance of cognitive levels to be tested.
• MEQs possess the potential to measure the full spectrum of cognitive abilities as defined by Bloom and Buckwalter.