CoMAD: a transferable method for building cognition-aware mathematics assessment datasets for educational AI research

Recent advances in Artificial Intelligence (AI), particularly large language models (LLMs), have accelerated research into automatic item generation, thereby fueling demand for curriculum-aligned, cognitively aware, contextualised datasets. However, there are limited cognitively annotated, contextually grounded, authentic assessment datasets that exhibit linguistic complexity and curriculum-aligned constructs. Moreover, dataset-construction methodologies struggle to integrate established educational taxonomies, linguistic complexity indicators, and FAIR-oriented data management into a single, transparent, and reproducible framework. We present the Cognition-Aware Mathematics Assessment Dataset construction (CoMAD), a transferable, FAIR-oriented framework for creating and curating higher-cognition, curriculum-aligned datasets augmented with computational data engineering. The method is validated using HiCogMath, a mathematics dataset in which cognitive annotations are based on Bloom's taxonomy and Webb's Depth of Knowledge, operationalised through Hess's Cognitive Matrix (CRM). CoMAD is a modular dataset-creation method that is scalable to other subjects, curricula, and contexts, creating datasets suitable for cognitive modelling, data mining, assessment analytics, automated item generation, Knowledge graph construction, and LLM alignment.•Transferable framework for construction of cognition-aware assessment datasets with curriculum alignment and linguistic profiling.•Presents transparent quality assurance for enhancing reliability, replicability and reusability of datasets.•Provides a modular framework that is transferable to diverse curricula, domains and contexts via annotation protocols.