Rasch analysis of distractors in multiple-choice items

Area of Science:

• Psychometrics
• Educational Measurement
• Item Response Theory

Background:

• Traditional Rasch model application for multiple-choice items aggregates incorrect responses, losing distractor-specific information.
• Analyzing individual distractor performance is crucial for effective item revision and improving assessment quality.

Purpose of the Study:

• To propose a Rasch-type analysis that assigns a unique parameter to each distractor, thereby preserving information.
• To introduce a distractor model that can diagnose distractor performance for item revision.
• To establish the relationship between the proposed distractor model and the standard Rasch model.

Main Methods:

• Development of a Rasch-type model with individual parameters for each distractor.
• Conducting a simulation study to evaluate parameter recovery for the distractor model.
• Analyzing a real dataset of twenty multiple-choice items using the proposed model.

Main Results:

• The proposed distractor model successfully preserves information lost in traditional Rasch analyses.
• Simulation results indicate highly satisfactory parameter recovery for the distractor model.
• Analysis of real data revealed some items fit the Rasch model but not the distractor model, highlighting diagnostic utility.

Conclusions:

• The proposed distractor model is a necessary condition for the Rasch model, offering enhanced diagnostic capabilities for multiple-choice items.
• This model provides valuable insights into distractor effectiveness, aiding in targeted item revision.
• The diagnostic value of the distractor model makes it a suitable tool for analyzing multiple-choice assessments.