Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Area of Science:

• Educational Measurement
• Psychometrics
• Assessment Design

Background:

• Single-choice items, including best-answer and true-false formats, have been standard in examinations for over a century.
• Accurate scoring of examinee responses is crucial for determining true knowledge levels.

Purpose of the Study:

• To systematically review and compile scoring methods for individual single-choice items.
• To analyze the expected chance score and the relationship between true knowledge and scoring results.
• To derive implications for setting appropriate pass marks in examinations.

Main Methods:

• A comprehensive literature search was conducted across multiple databases (ERIC, PsycInfo, Embase, MEDLINE) in September 2020.
• Scoring methods for single-choice items in written examinations were extracted, categorizing by the number of answer options (n=2 and n=5).
• Expected chance scores and expected scoring results were calculated for each method using simulated examinations.

Main Results:

• Twenty-one distinct scoring methods were identified from 258 sources, with variations in crediting correct, omitted, and incorrect answers.
• Expected chance scores for random guessing varied significantly based on the number of answer options, ranging from -1 to +0.75 for n=2 and -2.2 to +0.84 for n=5.
• Examination results varied substantially depending on the scoring method and number of answer options, with 50% true knowledge yielding scores from 0.0% to 92.0% across different scenarios.

Conclusions:

• Examination scores derived from single-choice items do not always equate to examinees' actual knowledge.
• Interpreting examination scores and establishing pass marks necessitates careful consideration of both the scoring method and the number of answer options per item.