Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Educational Technology

Background:

• Human action recognition is crucial for computer vision, particularly in educational settings.
• Existing research often focuses on single student behaviors, limiting comprehensive analysis.
• There is a need for robust systems to recognize diverse student actions in real-time.

Purpose of the Study:

• To develop an advanced student behavior recognition system using skeleton pose estimation and person detection.
• To enhance the accuracy and robustness of action recognition in classroom environments.
• To enable the identification of multiple students and their actions simultaneously.

Main Methods:

• Utilized consecutive frames from classroom cameras as input.
• Employed the OpenPose framework for skeleton data collection.
• Developed an error correction scheme integrating pose estimation and person detection.
• Performed feature extraction using normalized joint locations, joint distances, and bone angles.
• Implemented a deep neural network for behavior classification.

Main Results:

• The proposed system demonstrated superior performance compared to traditional skeleton-based methods in complex scenarios.
• Achieved a 15.15% increase in average precision and a 12.15% increase in average recall.
• Successfully identified the number of students present in the classroom.
• A system prototype confirmed the practical feasibility of the approach.

Conclusions:

• The novel system effectively recognizes student behaviors in classrooms by leveraging refined skeleton data.
• The integration of pose estimation and person detection significantly enhances recognition accuracy.
• This approach offers a promising solution for intelligent classroom monitoring and analysis.