Modeling with Real-Time Informative Feedback: Implementing and Evaluating a New Massive Open Online Course Component

As part of the design, development, and deployment of a massive open online course (MOOC) on model-based systems engineering, we introduced MORTIF-Modeling with Real-Time Informative Feedback, a new learning-by-doing feature that enables the learner to model, receive detailed feedback, and resubmit improved solutions. We examined the pedagogical usability of MORTIF by investigating characteristics of participants working with it, and their perceived contribution, preferred question type, and learning style. The research included 295 participants and applied the mixed-methods approach, using MOOC server data and online questionnaires. Analyzing 12,095 submissions, we found increasing frequency of using the model resubmitting option. Students ranked MORTIF as the highest of six question types in terms of preference and perceived contribution level. Nine learning style categories were identified and classified based on students' verbal explanations regarding their preference of MORTIF over the other question types. MORTIF has been effective in promoting meaningful learning, supporting our hypothesis that the combination of active learning with real-time informative feedback is a learning mode that students eagerly embrace and benefit from. The benefits we identified for using MORTIF include active learning, provision of meaningful immediate feedback to the learner, the option to use the feedback on the spot and resubmitting an improved model, and its suitability for a variety of learning styles.