Do Written Responses to Open-Ended Questions on Fourth-Grade Online Formative Assessments in Mathematics Help Predict...

Area of Science:

• Educational Data Mining
• Learning Analytics
• Psychometrics

Background:

• Predicting student achievement is crucial for educators and educational data mining.
• Real-world classrooms present challenges: teacher-generated questions and analyzing open-ended written responses.

Purpose of the Study:

• To develop a model predicting long-term student achievement considering unique classroom assessment data.
• To investigate the impact of analyzing open-ended student responses on prediction accuracy.

Main Methods:

• Analyzed 621,575 closed-ended and 16,618 open-ended answers from 464 fourth-graders in low socioeconomic status schools.
• Utilized linguistic features and an incoherent response classifier to build a linear prediction model.
• Predicted end-of-year national standardized test scores.

Main Results:

• Including features from students' open-ended responses improved the prediction of end-of-year test scores.
• This improvement was observed despite open-ended questions being answered less frequently than closed-ended ones.
• Demonstrated the value of analyzing written responses in online formative assessments.

Conclusions:

• Automatically analyzed features from open-ended responses enhance the prediction of student academic achievement.
• This study is the first to show improved score prediction by incorporating features from open-ended online formative assessment answers.
• Highlights the potential of leveraging written student responses for more accurate educational outcome predictions.