Subjective data, objective data and the role of bias in predictive modelling: Lessons from a dispositional learning...

Area of Science:

• Educational research
• Learning analytics
• Psychometrics

Background:

• Self-report questionnaires are traditional tools for measuring complex learning constructs.
• Potential biases in self-report data raise concerns about construct validity.
• Trace data from digital learning environments offer an alternative, potentially objective, measurement approach.

Purpose of the Study:

• To investigate the strengths and weaknesses of self-report and trace data for predicting academic performance.
• To examine response style and overconfidence biases in self-report surveys.
• To compare the predictive validity of different data types in educational modeling.

Main Methods:

• Analysis of computer-generated trace data and survey data.
• Investigation of response style bias (systematic rating scale use) and overconfidence bias (discrepancy between predicted and actual performance).
• Development of predictive models for academic performance using various data sources.

Main Results:

• Response style bias significantly influences self-report instruments and course performance data.
• Process-type trace data demonstrate independence from response style patterns.
• Overconfidence bias has a limited effect on predictive models.

Conclusions:

• Trace data, particularly process data, offer a more objective measure for learning analytics.
• Survey biases, while problematic for construct validity, can add predictive power to educational models.
• Integrating diverse data sources, including trace data, is crucial for robust academic performance prediction.