A machine learning-based procedure for leveraging clickstream data to investigate early predictability of failure on...

Area of Science:

• * Educational Technology
• * Human-Computer Interaction
• * Machine Learning

Background:

• * Early detection of failure risk in interactive tasks enhances understanding of examinee behavior and allows for adaptive task adjustments.
• * Clickstream data, commonly used in e-commerce for predicting shopper intent, offers a novel approach for analyzing user behavior in educational or assessment contexts.

Purpose of the Study:

• * To introduce and demonstrate a machine learning procedure for predicting failure risk in interactive tasks using early clickstream data.
• * To investigate the early predictability of behavioral outcomes by analyzing features derived from initial user actions.

Main Methods:

• * Leveraged early-window clickstream data (occurrence, frequency, sequentiality, timing of actions) from interactive tasks.
• * Employed extreme gradient boosting (XGBoost) for classification of success and failure.
• * Evaluated prediction quality using multiple performance measures.

Main Results:

• * Achieved significant prediction performance, identifying at-risk examinees when they were, on average, two-thirds through the solution process.
• * Demonstrated that the vast majority of examinees who ultimately failed could be flagged as potentially at risk before task completion.
• * Identified distinct behavioral features indicative of success versus failure at different stages of task engagement.

Conclusions:

• * The proposed machine learning procedure effectively predicts failure risk in interactive tasks using early behavioral data.
• * This method offers a valuable tool for adaptive testing and a deeper understanding of examinee behavior trajectories.
• * Findings highlight the potential for early intervention and personalized learning experiences based on real-time behavioral analysis.