Explainable AI Framework for Accuracy, Fairness, and Learner Perception in English Writing Assessment.

Area of Science:

• Artificial Intelligence in Education
• Natural Language Processing
• Educational Technology

Background:

• Automated Writing Evaluation (AWE) offers efficiency but often lacks transparency and equity.
• Traditional AWE frameworks may neglect learner perceptions and fairness, limiting educational value.
• Explainable AI (XAI) is crucial for trust and understanding in educational tools.

Purpose of the Study:

• To propose and validate an explainable AI (XAI) framework for Automated Writing Evaluation (AWE).
• To integrate a multi-level validation model (Three-Level Evaluation Framework - TLEF) addressing accuracy, equity, and perception.
• To investigate fairness biases and learner perceptions in AWE across multilingual learners.

Main Methods:

• Collected data from 764 multilingual learners (CEFR A2-C1) using writing tasks and questionnaires.
• Employed stratified random sampling and dual AI/human expert scoring.
• Utilized statistical analyses including correlation, RMSE, Equalized Odds, and SEM.

Main Results:

• The AWE system showed overall validity (r = 0.82) but revealed significant disparities.
• Chinese native speakers exhibited lower agreement (0.72) and higher RMSE (median 2.15).
• Fairness biases were more pronounced at lower proficiency levels (A2 learners), and perceived fairness mediated learner satisfaction.

Conclusions:

• Reframing fairness and perception as key explainability dimensions enhances AWE's theoretical foundation.
• The proposed XAI framework offers a practical approach to improve transparency, equity, and social acceptance in educational technologies.
• Addressing AI fairness and learner perception is vital for advancing AWE in digital education transformation.