Crowdsourcing for assessment items to support adaptive learning.

Area of Science:

• Medical Education
• Assessment Science

Background:

• Adaptive learning necessitates frequent, valid assessments for learner progress tracking.
• The efficacy of crowdsourced multiple-choice questions (MCQs) in meeting stringent assessment standards is under investigation.

Purpose of the Study:

• To evaluate if MCQs crowdsourced from medical learners meet the rigorous standards of large-scale testing programs.
• To assess the quality and statistical validity of crowdsourced assessment items.

Main Methods:

• Medical learners submitted case-based MCQs through an educational app.
• A faculty panel reviewed 200 MCQs for relevance, accuracy, and explanation quality.
• A subset of 121 MCQs was pretested on U.S. medical students.

Main Results:

• 78% of crowdsourced MCQs met faculty reviewer standards for quality and accuracy.
• 50% of pretested MCQs met acceptable statistical criteria, with common exclusions being items that were too easy or had low discrimination.
• Crowdsourcing demonstrated efficiency in generating assessment items.

Conclusions:

• Crowdsourcing effectively produces high-quality assessment items that satisfy both judgmental and statistical criteria.
• This model offers a viable strategy for students and educators to expand item pools for adaptive learning systems.