Multimodal Teaching Analytics: Automated Extraction of Orchestration Graphs from Wearable Sensor Data

Area of Science:

• Educational Technology
• Machine Learning in Education
• Classroom Practice Analysis

Background:

• Pedagogical modeling provides valuable insights for educational research and teacher professional development.
• Automating the analysis of classroom practice can enhance the efficiency and scalability of educational research.

Purpose of the Study:

• To explore the use of wearable sensors and machine learning for automatically extracting orchestration graphs (teaching activities and their social plane over time).
• To evaluate the feasibility of automated classroom practice tagging for use in teacher professional development and educational research.

Main Methods:

• Collected data from 12 classroom sessions using wearable sensors (mobile eye-tracking, audiovisual, accelerometry) worn by teachers.
• Applied both time-independent and time-aware machine learning models to analyze sensor data.
• Utilized a leave-one-session-out k-fold cross-validation strategy for model evaluation.

Main Results:

• Achieved median F1 scores of approximately 0.7-0.8 in automatically extracting orchestration graphs.
• Demonstrated the feasibility of using wearable sensor data and machine learning for analyzing pedagogical practices.
• Identified limitations including the need for larger, more diverse datasets for broader applicability.

Conclusions:

• Automated tagging of classroom practice using wearable sensors and machine learning is feasible.
• Further research with larger and more varied datasets is necessary to generalize findings across different classroom settings and teachers.
• This technology holds potential for supporting teachers' professional development and advancing educational research.