Understanding Adults' Spatial Cognitive Processes: A Time-Embedded N-Grams Model with Machine Learning

Technological advances have transformed the landscape of educational assessments, particularly in how we collect and analyze data on individuals' cognitive processes and interactions with assessment items. The rich data recorded in log files during human-machine interactions are often referred to as process data. This study uses sequential process data from the 2012 Program for the International Assessment of Adult Competencies (PIAAC), focusing specifically on how respondents navigate an interactive numeracy item ("Map") that involves spatial cognition. The objectives of this study are trifold: (1) to explore factors that contribute to success or failure on a spatial numeracy item, (2) to identify spatial cognitive process features across high and low numeracy performance levels, and (3) to introduce a novel time-embedded n-grams model to incorporate elapsed time with sequential actions in process data analysis. Using a sample of 596 U.S. adult respondents, we employed the time-embedded n-grams model and two machine learning methods, random forest and XGBoost, to predict respondents' numeracy performance and to identify robust classifiers. Results indicate that time-related features and understanding directions in the spatial dimensions are the most predictive of adults' numeracy skills. The findings highlight the potential of process data to support analyses of latent numeracy skills and cognitive processes in educational contexts.