Real-Time Four-Category Student Engagement Classification from a Single Webcam Using a Hybrid ResNet and Geometric Feature Network

The Engagement Insights system is an automated technology that tracks student engagement in real-time with the help of just one webcam which classifies engagement into four main categories, namely, Attentive, Confused, Disengaged, and Distracted. In comparison to the usual methods that are based on binary definitions and needed special devices that are often impractical (like wearable devices), the system provides an easy and non-intrusive alternative. The system employs ResNet convolutional neural networks alongside the geometry module that is capable of calculating Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR) using facial landmark detection framework facial points which enhance reliability while remaining speedy. The system has been trained and validated using the dataset that comprises pictures from AffectNet and video frames from the DAiSEE database that have all been labeled into the four engagement categories with the help of educational psychology patterns and independent evaluations showing that the achieved Cohen's kappa coefficient equals 0.87. The system worked well as the average classification accuracy was 89% with the average macro-F1 score being 0.889 and AUC ranging from 0.926 to 0.936 for all the classes. Most importantly, the analysis of the dataset has shown that by using ImageNet weights, it is possible to increase classification performance by 13-17 percentage points in comparison to training without pre-trained weights which confirms the findings made in the main text. Moreover, the geometric module of the system did increase the performance in detecting disengagement signals but did not slow down performance since it only took 2 milliseconds to process one frame which enables the system to work independently on regular laptops providing the necessary number of frames per second while requiring no additional expensive equipment like GPUs.