Analyzing Complex Educational Data: A Data Analytic Framework for Integrating Structured and Unstructured...

Area of Science:

• Educational Technology
• Psychometrics
• Cognitive Science

Background:

• Computer-based assessments generate complex, real-time process data.
• Eye-tracking data offers rich temporal insights into visual information processing during problem-solving.
• Analyzing high-dimensional, multimodal, and temporally dependent data presents significant methodological challenges.

Purpose of the Study:

• To introduce a two-component data-analytic framework (DAK) for integrating and interpreting structured and unstructured data in educational assessments.
• To extract latent features representing dynamic visual attention patterns from eye-tracking data.
• To generate construct-relevant validity evidence for test-taking and learning behaviors using integrated multimodal data.

Main Methods:

• Developed a time-aware long short-term memory Autoencoder incorporating fixation duration and elapsed time.
• Employed a data-driven temporal decay function and optimized a multi-target reconstruction objective.
• Integrated extracted features using clustering, categorical data analyses, and mixed-effects modeling.

Main Results:

• Demonstrated the DAK using spatial rotation learning program data (structured scores and eye-tracking).
• Identified distinct behavioral patterns associated with test performance.
• Revealed behavioral patterns linked to intervention effectiveness.

Conclusions:

• The DAK effectively integrates multimodal process data for advanced psychometric modeling.
• Eye-tracking and structured data analysis can reveal critical insights into learning and assessment behaviors.
• This approach holds significant potential for improving psychometric modeling and educational instrument design.