A new person-fit method based on machine learning in CDM in education

Area of Science:

• Educational Measurement
• Psychometrics
• Machine Learning

Background:

• Cognitive diagnosis models (CDMs) offer detailed student feedback beyond traditional scores.
• Data biases and aberrant response patterns can compromise CDM accuracy.
• Existing person-fit methods may lack sufficient power to detect these issues.

Purpose of the Study:

• To develop a novel machine learning-based person-fit method.
• To enhance the accuracy of cognitive diagnosis models by addressing aberrant response patterns.
• To improve upon the effectiveness of existing person-fit techniques.

Main Methods:

• Development of a new person-fit methodology utilizing machine learning algorithms.
• Simulations conducted under three aberrant conditions: cheating, sleeping, and random guessing.
• Comparative analysis against existing person-fit methods.

Main Results:

• The new machine learning person-fit method demonstrated superior power and effectiveness.
• The method showed particular efficacy in detecting aberrant patterns in short-length tests.
• Improved accuracy in identifying individual student response behaviors was observed.

Conclusions:

• The proposed machine learning person-fit method offers a significant advancement for cognitive diagnosis.
• This approach enhances the reliability of individualized feedback in educational assessments.
• The method provides a robust tool for detecting problematic response behaviors, even with limited test data.