Theory of change models deepen online learning evaluation.

Area of Science:

• Educational Technology
• Professional Development Evaluation
• Online Learning Research

Background:

• Massive Open Online Courses (MOOCs) are increasingly used for professional development.
• Traditional MOOC evaluations often focus on participation and completion rates.
• There is a need for deeper understanding of learning processes and long-term outcomes in MOOCs.

Purpose of the Study:

• To present a novel method for evaluating professional development MOOCs.
• To guide MOOC designers and evaluators in focusing on the learning process.
• To complement traditional MOOC evaluation approaches.

Main Methods:

• Developed a theory of change for each MOOC module in collaboration with content creators.
• Utilized the theory of change to evaluate short- and medium-term cognitive, affective, and behavioral outcomes.
• Applied this approach to a professional development MOOC for postdoctoral trainees.

Main Results:

• The theory of change approach provided a framework for understanding the learning process within MOOCs.
• Enabled evaluation of a wider range of participant outcomes beyond completion rates.
• Offered insights into how learning translates to cognitive, affective, and behavioral changes.

Conclusions:

• Theories of change offer a valuable framework for designing and evaluating MOOCs.
• This method enhances the depth of MOOC evaluation by focusing on the learning journey.
• Recommendations are provided for implementing theories of change in MOOC development and assessment.