Measuring growth in students' proficiency in MOOCs: Two component dynamic extensions for the Rasch model

Area of Science:

• Educational Measurement
• Psychometrics
• Online Learning

Background:

• Massive Open Online Courses (MOOCs) are widely adopted globally.
• Accurate measurement of student proficiency growth is crucial for students, professors, and institutions.
• Traditional Item Response Theory (IRT) models assume static proficiency, limiting growth measurement.

Purpose of the Study:

• To propose and validate dynamic extensions for measuring two components of proficiency growth in MOOCs.
• To address the limitations of traditional IRT models in capturing temporal changes in student ability.
• To enhance the assessment of learning progression within online educational environments.

Main Methods:

• Developed two dynamic extensions of the Rasch model, a common IRT model.
• Applied these extensions to analyze logged data from three real MOOCs.
• Utilized a cross-validation procedure to assess the quality and predictive accuracy of the proposed models.

Main Results:

• Student proficiency was found to grow both across entire courses and within specific learning objectives.
• The dynamic extensions demonstrated a superior fit to the data compared to the original Rasch model.
• The proposed models achieved an average accuracy of .763 in predicting student responses.

Conclusions:

• Dynamic IRT models offer a more accurate approach to measuring proficiency growth in MOOCs.
• The developed extensions effectively capture within-course and within-learning-objective proficiency gains.
• These findings have significant implications for the assessment and understanding of learning in online education.