Question format is the best predictor of item discrimination: a multivariable analysis

Area of Science:

• Educational Assessment
• Biology Education Research
• Psychometrics

Background:

• Item discrimination is crucial for effective assessment, measuring how well an item differentiates between high and low performers.
• Few studies jointly analyze the impact of question format, difficulty, and cognitive level (Bloom's Taxonomy) on item discrimination.

Purpose of the Study:

• To investigate the independent contributions of question format, difficulty, and Bloom's Taxonomy level to item discrimination in undergraduate biology examinations.
• To identify optimal question characteristics for enhancing the quality of biological assessments.

Main Methods:

• Analysis of a biology item bank (n=754) using simultaneous multivariable linear and logistic regression models.
• Predictor variables included question format (MCQ, SA, TF), difficulty (easy, moderate, hard), and Bloom's Taxonomy level (recall, understanding, application, analysis).

Main Results:

• Question format was the strongest independent predictor of item discrimination.
• Short-answer (SA) items demonstrated significantly higher discrimination than multiple-choice questions (MCQs).
• Moderate and hard difficulty items, as well as higher-order Bloom's levels (understanding, analysis), showed better discrimination than easy items and recall questions, respectively.

Conclusions:

• Question format, particularly the use of short-answer questions, is a key factor in improving item discrimination.
• Incorporating moderate difficulty and higher-order cognitive tasks (understanding, analysis) in assessments can enhance their quality.
• A pragmatic strategy for improving biology assessment quality involves using higher-order short-answer questions of moderate difficulty.