Related Experiment Videos
Factors underlying performance on written tests of knowledge.
Medical Education
|July 1, 1987
Summary
This study found that while multiple choice questions (MCQ) and Modified Essay Questions (MEQ) assess different skills, their content scores are highly correlated. Question context significantly impacts performance, with problem-based learning outperforming random sequencing.
Area of Science:
- Medical Education
- Assessment Methods
- Educational Psychology
Background:
- Written tests are crucial for evaluating medical student knowledge.
- Multiple Choice Questions (MCQ) and Modified Essay Questions (MEQ) are widely used assessment formats.
- Understanding the comparability and distinctiveness of these formats is essential for effective assessment design.
Purpose of the Study:
- To compare the performance of medical students on Multiple Choice Questions (MCQ) and Modified Essay Questions (MEQ).
- To investigate the influence of question format (MCQ, directed free response, open-ended free response) and context (patient problem vs. random sequence) on test scores.
- To assess the correlation between content scores across different question formats.
Main Methods:
- A Latin-square design was employed to administer six unique problems in each of three question formats (MCQ, directed free response, open-ended free response) and two contexts (patient problem, random sequence).
- Thirty-six medical students across three educational levels participated in the study.
- Scores were analyzed to determine the effects of question format and context.
Main Results:
- Significant differences in scores were observed based on question format, with MCQs yielding higher scores than directed free response (8.7% difference), and directed free response outperforming open-ended free response (4.2% difference).
- Placing questions within a patient problem context resulted in significantly better performance (4.3% difference) compared to random sequencing.
- Despite format differences, content scores across MCQ and MEQ formats showed a strong correlation, approaching unity after correction.
- Process scores, evaluating presentation style in free-response formats, correlated more strongly with free-response questions.
Conclusions:
- Multiple Choice Questions and Modified Essay Questions, while potentially assessing different skills, demonstrate a strong congruence in evaluating content knowledge.
- Free-response formats offer opportunities to assess additional skills like presentation and style, provided scoring methods are adapted.
- Presenting assessment questions within a relevant patient problem context enhances student performance compared to random question ordering.