Model-agnostic counterfactual reasoning for identifying and mitigating answer bias in knowledge tracing

Area of Science:

• Artificial Intelligence
• Educational Technology
• Machine Learning

Background:

• Knowledge tracing (KT) models student knowledge states from learning interactions.
• Existing KT models often rely on answer bias, hindering accurate knowledge state assessment.
• Answer bias, an unbalanced distribution of correct/incorrect answers, is a common issue in educational datasets.

Purpose of the Study:

• To address the problem of answer bias in knowledge tracing.
• To develop a method that prevents models from exploiting answer bias as a shortcut.
• To improve the accuracy of student knowledge state monitoring.

Main Methods:

• Established a causal graph for knowledge tracing.
• Proposed a novel COunterfactual REasoning (CORE) framework.
• CORE separates total and direct causal effects to mitigate answer bias during inference.

Main Results:

• Demonstrated the effectiveness of the CORE framework across multiple benchmark datasets.
• CORE successfully debiased KT inference, leading to more accurate student knowledge state estimations.
• The framework was successfully integrated with existing models like DKT, DKVMN, and AKT.

Conclusions:

• The causal perspective offers a robust solution to answer bias in knowledge tracing.
• CORE framework enhances the reliability and interpretability of knowledge tracing models.
• This approach provides a significant advancement for personalized learning systems.