Applying a Learning Analytics Lifecycle Framework to Identify Gaps in Digital Learning Design: A Worked Example in...

Learning analytics involve collecting, analyzing, and visualizing the digital 'footprints' that learners leave behind as they interact with digital learning environments. Learning analytics inform ongoing refinements to improve educational design and teaching practice. To date, research suggests that educators have not fully capitalized on learning analytics, despite the growing availability of data generated through digital learning platforms, programmatic assessment, and competency-based training. In this article, the authors adapted an established "learning analytics lifecycle" framework to evaluate a representative example of digital learning in HPE: eduCAST, an online, video-based website for orthopaedic surgery training. The framework provided a stepwise approach for exploring the potential of learning analytics in eduCAST, identifying implementation gaps, and developing a transferable list of recommendations for educators in HPE. Demographic and engagement data of 141 registered eduCAST users were analyzed after being collected via website analytics. The findings revealed limitations and opportunities related to: (1) planning for the learning environment and its users, (2) the scope and specificity of available learning analytics data, (3) the purposeful use of data analysis techniques, and (4) the relative absence of educators' data-informed actions. Generic website analytics showed significant limitations: high website traffic did not correspond to meaningful learner engagement, nor did it provide useful data to inform ongoing website refinement. Indeed, these analytics produced more data mysteries than meaningful data stories. Consistent with prior reviews in the HPE literature, this worked example demonstrated that educators appear to make diminishing investments of effort and resources across the learning analytics lifecycle. The authors argue that HPE educators can benefit from using learning analytics frameworks to guide the design, implementation, evaluation, and long-term sustainability of digital learning environments. When thoughtfully collected, analyzed, and interpreted, learning analytics can enhance learners' experiences and outcomes, support educators' professional development, and inform continuous program refinement.