Bimodal Learning Engagement Recognition from Videos in the Classroom

Area of Science:

• Artificial Intelligence in Education
• Computer Vision for Learning Analytics

Background:

• Learning engagement is crucial for understanding student states and optimizing education.
• Traditional engagement recognition methods (self-report, observation) are impractical for large classrooms.
• Deep learning offers a promising avenue for automated engagement recognition.

Purpose of the Study:

• To develop an automated, non-invasive method for recognizing learning engagement in classrooms.
• To construct a multi-cues database for learning engagement analysis.
• To evaluate the efficacy of a novel bimodal recognition approach.

Main Methods:

• Constructed a multi-cues classroom learning engagement database from non-invasive videos.
• Utilized You Only Look Once version 5 (YOLOv5) with power IoU loss for student detection (95.4% precision).
• Designed a bimodal learning engagement recognition model using ResNet50 and CoAtNet, classified with KNN.

Main Results:

• Achieved 95.4% precision in student detection using YOLOv5 with power IoU loss.
• The bimodal recognition method attained 93.94% accuracy with a KNN classifier.
• Experimental results demonstrate superior performance compared to state-of-the-art methods.

Conclusions:

• The proposed bimodal learning engagement recognition method is effective and accurate.
• AI-based analysis of classroom videos offers a scalable solution for monitoring student engagement.
• This research advances intelligent methods for educational data mining and learning analytics.