Evaluation of five guidelines for option development in multiple-choice item-writing

Area of Science:

• Educational Measurement
• Psychometrics

Background:

• Effective multiple-choice test item writing is crucial for accurate assessment of student knowledge.
• Previous guidelines suggested avoiding certain item characteristics, such as varying option lengths or content heterogeneity.

Purpose of the Study:

• To empirically evaluate common guidelines for writing multiple-choice test items.
• To determine the impact of specific item characteristics on test performance and discrimination.

Main Methods:

• Analysis of response data from 5013 subjects across 630 multiple-choice items from 21 university achievement tests.
• Statistical examination of item discrimination based on variations in option content, length, and the use of 'None of the above' or 'All of the above' options.

Main Results:

• Heterogeneous content among options negatively impacts item discrimination.
• The 'None of the above' option, when correct, also shows a detrimental effect on discrimination.
• No significant negative effects were found for differing option lengths or the use of specific determiners.
• The 'All of the above' option, when correct, decreased item difficulty and improved discrimination.

Conclusions:

• Educators should avoid heterogeneous content in multiple-choice options to enhance item quality.
• The 'All of the above' option can be a beneficial strategy for improving item discrimination in achievement tests.