Classification of Internal and External Distractions in an Educational VR Environment Using Multimodal Features

Area of Science:

• Educational Technology
• Neuroscience
• Human-Computer Interaction

Background:

• Virtual reality (VR) offers potential for enhanced student engagement and memory retention.
• Distraction, both internal and external, poses a significant challenge to learning in VR environments.
• Single-modal features are insufficient for accurately detecting diverse distractions due to individual variability.

Purpose of the Study:

• To investigate the effectiveness of multi-modal features (eye tracking and EEG) for classifying internal and external distractions in an educational VR setting.
• To compare the performance of various machine learning models in detecting these distractions.
• To identify crucial features contributing to distraction detection.

Main Methods:

• An educational VR environment was established for multi-modal data collection (eye tracking and EEG).
• Machine learning models including kNN, Random Forest (RF), 1D-CNN-LSTM, and 2D-CNN were implemented.
• Models were evaluated using cross-subject, cross-session, and gender-based grouping tests.

Main Results:

• The Random Forest classifier demonstrated the highest accuracy, exceeding 83% in cross-subject tests.
• Cross-session accuracy ranged from 68% to 78%, while gender-based grouping achieved approximately 90% accuracy.
• SHAP analysis highlighted the importance of occipital and prefrontal brain regions, gaze angle, gaze origin, and head rotation in distraction detection.

Conclusions:

• Multi-modal data fusion, particularly with eye tracking and EEG, is effective for classifying distractions in educational VR.
• The Random Forest model shows strong performance in identifying student distraction states.
• Understanding the contribution of specific neurophysiological and eye-movement features can inform the design of more engaging and less distracting VR learning experiences.