Multiple-Choice Item Distractor Development Using Topic Modeling Approaches.

Area of Science:

• Educational Assessment
• Biology Education

Background:

• Developing high-quality multiple-choice items is challenging, particularly in creating plausible distractors.
• Current methods often lack systematic approaches, leading to less effective distractors.

Purpose of the Study:

• To describe and demonstrate a systematic method for generating plausible distractors for multiple-choice questions.
• To base distractor creation on identified student misconceptions.

Main Methods:

• Utilized latent Dirichlet allocation (LDA), a topic modeling technique from machine learning and natural language processing.
• Analyzed 1,515 Grade 10 Biology student written responses to a constructed-response item.
• Identified 22 plausible misconceptions to inform distractor generation.

Main Results:

• Successfully extracted 22 distinct student misconceptions from written responses.
• Developed plausible distractors directly derived from these identified misconceptions.
• Integrated these distractors into new multiple-choice test items.

Conclusions:

• The described systematic method provides a data-driven approach to creating effective distractors.
• Leveraging student misconceptions enhances the quality and diagnostic power of multiple-choice assessments.
• This method offers valuable implications for educational item development and assessment design.