Taking a Closer Look: An Exploratory Analysis of Successful and Unsuccessful Strategy Use in Complex Problems

Influencing students' educational achievements first requires understanding the underlying processes that lead to variation in students' performance. Researchers are therefore increasingly interested in analyzing the differences in behavior displayed in educational assessments rather than merely assessing their outcomes. Such analyses provide valuable information on the differences between successful and unsuccessful students and help to design appropriate interventions. Complex problem-solving (CPS) tasks have proven to provide particularly rich process data as they allow for a multitude of behaviors several of which can lead to a successful performance. So far, this data has often been analyzed on a rather aggregated level looking at an average number of actions or predefined strategies with only a few articles investigating the specific actions performed. In this paper, we report the results of an exploratory analysis of CPS log-files that is aimed at distinguishing between students that applied the correct strategy to a problem but failed to solve it and those applying the strategy successfully. In that, the sequence of behavior displayed is reduced to interpretable parts (n-grams) that allow searching for meaningful differences between the two groups of students. This level of analysis allows finding previously undefined or unknown patterns within the data and increases our understanding of the processes underlying successful problem-solving behavior even further.