Assessing Comprehensive Spatial Ability and Specific Attributes Through Higher-Order LLM

Spatial reasoning ability plays a critical role in predicting academic outcomes, particularly in STEM (science, technology, engineering, and mathematics) education. According to the Cattell-Horn-Carroll (CHC) theory of human intelligence, spatial reasoning is a general ability including various specific attributes. However, most spatial assessments focus on testing one specific spatial attribute or a limited set (e.g., visualization, rotation, etc.), rather than general spatial ability. To address this limitation, we created a mixed spatial test that includes mental rotation, object assembly, and isometric perception subtests to evaluate both general spatial ability and specific attributes. To understand the complex relationship between general spatial ability and mastery of specific attributes, we used a higher-order linear logistic model (HO-LLM), which is designed to simultaneously estimate high-order ability and sub-attributes. Additionally, this study compares four spatial ability classification frameworks using each to construct Q-matrices that define the relationships between test items and spatial reasoning attributes within the HO-LLM framework. Our findings indicate that HO-LLMs improve model fit and show distinct patterns of attribute mastery, highlighting which spatial attributes contribute most to general spatial ability. The results suggest that higher-order LLMs can offer a deeper and more interpretable assessment of spatial ability and support tailored training by identifying areas of strength and weakness in individual learners.