Automated Disengagement Tracking Within an Intelligent Tutoring System

Area of Science:

• Educational Technology
• Artificial Intelligence in Education
• Human-Computer Interaction

Background:

• Intelligent tutoring systems (ITS) aim to enhance learning outcomes.
• Detecting learner disengagement is crucial for effective adaptive learning.
• Previous methods for disengagement detection often rely on self-reports, which can be unreliable.

Purpose of the Study:

• To introduce and validate an automated disengagement tracking system (DTS) for intelligent tutoring systems.
• To assess the effectiveness of DTS in identifying maladaptive behaviors like mind-wandering and impetuous responding.
• To examine the relationship between detected engagement levels and reading comprehension performance.

Main Methods:

• Developed an unsupervised learning-based DTS to detect disengagement in AutoTutor, a conversational ITS for adult literacy.
• Established a baseline performance using initial response accuracy and time.
• Identified disengagement by detecting significant deviations from the established baseline.

Main Results:

• The DTS identified disengaged items with significantly lower accuracy (18.5%) compared to engaged items (71.8%).
• Learner response time and accuracy were analyzed for 252 adult literacy learners.
• Reading comprehension scores were significantly associated with engaged item accuracy, but not disengaged item accuracy.

Conclusions:

• The automated disengagement tracking system effectively distinguishes between engaged and disengaged learning states.
• Disengagement significantly impacts performance accuracy within the intelligent tutoring system.
• Accurate engagement detection is vital for reliable assessment of reading comprehension skills in ITS.