Understanding and Predicting Change Points in Individual Learning: An Analysis With Real-World Data

Different models of learning have been developed to account for human performance over time, often fitting aggregate rather than individual performance. Understanding performance at an individual level is often more difficult because multiple factors (e.g., motivation, strategy exploration, or changes in memory representation) vary across individuals and can lead to sudden changes in performance that cannot be accounted for by models used to account for aggregate behavior. One approach used to account for the sudden changes in individual performance is to integrate change detection algorithms (CDAs) with learning models. Indeed, past research has shown that performance at the individual level can be understood not as a single continuous process but instead as a sequence of different segments marked by change points, each accounted for by different or varying processes. Previous research has posited different explanations as to what features (e.g., individual, problem, problem-type) drive the inferences of these change points. Here, we compared these different explanations' ability to explain the variance in inferred change points across a single data set. To this end, we use a simple model of learning paired with a CDA to account for performance in a real-world data set with individuals performing multiple different games on the website Lumosity that tap into different task attributes (i.e., memory, attention, problem-solving, etc.). Our findings confirmed previous results about the nature of individual learning curves proposed from prior research. We found that change points were inferred across a majority of the users' game performance and that the inferred change points were associated with greater overall performance improvement over time. Finally, we found that the task features previously proposed to be associated with change points could both explain and predict inferred change points to varying degrees. The results clarify what features are driving the inferences of change points and show how psychological theory can be used to better understand and predict real-world behavior at the individual level.