Data Mining Techniques in Analyzing Process Data: A Didactic.

Area of Science:

• Educational Measurement
• Data Mining
• Machine Learning

Background:

• Technology-enhanced educational assessments generate extensive process data in log files.
• Previous research often focused on single data mining techniques in isolation.
• A comprehensive comparison of various data mining methods for educational assessment data is needed.

Purpose of the Study:

• To demonstrate and compare the effectiveness of multiple supervised and unsupervised data mining techniques for analyzing educational assessment process data.
• To provide guidance on selecting appropriate data mining classifiers based on research needs.
• To interpret findings from both supervised and unsupervised learning approaches.

Main Methods:

• Utilized four supervised learning techniques: Classification and Regression Trees (CART), gradient boosting, random forest, and Support Vector Machine (SVM).
• Employed two unsupervised learning methods: Self-organizing Map (SOM) and k-means clustering.
• Applied these methods to process data from the 2012 Program for International Student Assessment (PISA) USA sample (N=426) problem-solving items, including feature generation and selection.

Main Results:

• All implemented supervised and unsupervised data mining techniques demonstrated satisfactory classification accuracy.
• The study successfully applied and interpreted results from diverse machine learning algorithms on assessment data.
• Feature generation and selection were crucial steps for effective classifier development.

Conclusions:

• Multiple data mining techniques, both supervised and unsupervised, are effective for analyzing educational assessment log data.
• Classifier selection should consider research questions, interpretability, and simplicity.
• The findings offer valuable insights for researchers utilizing data mining in educational assessment.