Middle school teachers' implementation and perceptions of automated writing evaluation

Area of Science:

• Educational Technology
• Writing Instruction
• Human-Computer Interaction

Background:

• Automated Writing Evaluation (AWE) systems show promise for improving student writing.
• Teacher implementation of AWE systems is often inconsistent, impacting real-world effectiveness.
• Factors influencing teachers' adoption and perceptions of AWE are not well understood.

Purpose of the Study:

• To investigate the relationship between teachers' implementation and perceptions of the MI Write AWE system.
• To identify actionable factors that can enhance future AWE implementation and acceptance.
• To understand the contextual elements influencing AWE use in educational settings.

Main Methods:

• A mixed-methods approach combining quantitative (usage logs, surveys) and qualitative (interviews, focus groups) data.
• Analysis of data from 19 teachers participating in a randomized controlled trial (RCT) of MI Write.
• Descriptive and non-parametric statistical analyses for quantitative data; deductive coding for qualitative data.

Main Results:

• Teachers showed variable implementation of MI Write, below expected levels in the RCT.
• Teachers generally held positive attitudes towards MI Write, but perceived usability and usefulness were insufficient for effective implementation.
• Ecological factors, including curriculum alignment, workload integration, administrative support, and policy, significantly influenced implementation.

Conclusions:

• Effective implementation of AWE systems like MI Write depends on more than just system functionality.
• Addressing broader ecological and contextual factors is crucial for successful AWE adoption and integration.
• Understanding these factors can inform strategies to improve AWE's practical impact on writing instruction.