Psychometrics of MOOCs: Measuring Learners' Proficiency

Area of Science:

• Educational Measurement
• Psychometrics
• Learning Analytics

Background:

• Massive Open Online Courses (MOOCs) generate extensive learner performance data.
• Current assessment methods in MOOCs are often oversimplified, relying on basic metrics like correct response proportions.
• These simplistic approaches fail to capture the complexity of learner proficiency due to unconsidered learner, content, and contextual factors, potentially leading to flawed conclusions.

Purpose of the Study:

• To bridge the gap between psychometrics and MOOCs.
• To highlight the need for theoretically grounded, data-driven, and explainable educational measurement in MOOCs.
• To explore how psychometric principles can be applied to analyze MOOC data more effectively.

Main Methods:

• Conceptual paper outlining principles of traditional educational measurement.
• Discussion of MOOC-specific characteristics that challenge conventional measurement techniques.
• Review of recent advancements in proficiency measurement relevant to MOOC data analysis.
• Identification of potential psychometric modeling directions for future MOOC research.

Main Results:

• Traditional educational measurement principles are described.
• Limitations of applying traditional methods directly to MOOCs are identified.
• Relevant recent developments in proficiency measurement are discussed.
• Future research directions in psychometric modeling for MOOCs are proposed.

Conclusions:

• Existing methods for measuring learner proficiency in MOOCs are inadequate.
• There is a critical need for advanced, psychometrically informed approaches to analyze MOOC data.
• Future research should focus on developing sophisticated psychometric models tailored for the unique environment of MOOCs.