Metaverse-Powered Experiential Situational English-Teaching Design: An Emotion-Based Analysis Method

Area of Science:

• Educational Technology
• Neuroscience
• Computer Science

Background:

• The metaverse offers immersive and interactive educational experiences, advancing visual immersion technology.
• Digital technologies like VR, AR, big data, and 5G enable virtual worlds for education.
• The metaverse provides a parallel digital space for innovation in human society and professions.

Purpose of the Study:

• To construct an experiential situational English-teaching scenario within the metaverse.
• To develop and evaluate Convolutional Neural Networks (CNNs)-Recurrent Neural Networks (RNNs) fusion models for recognizing student emotions.
• To analyze the impact of metaverse-powered English teaching on student interactivity, immersion, and cognition.

Main Methods:

• Designed three types of experiential English-teaching scenarios: sequential guidance, comprehensive exploration, and crowd-creation construction.
• Collected electroencephalogram (EEG) data using an OpenBCI EEG Electrode Cap Kit.
• Utilized CNN-RNN fusion models to analyze EEG data across time, frequency, and spatial domains for emotion recognition.

Main Results:

• Metaverse-powered English teaching significantly improved students' sense of interactivity, immersion, and cognition.
• The CNN-RNN fusion model demonstrated higher accuracy and faster analysis time compared to baseline models.
• The study successfully recognized student emotions during experiential English teaching in a metaverse setting.

Conclusions:

• Metaverse-enhanced English teaching positively impacts student engagement and learning outcomes.
• Advanced AI models like CNN-RNN fusion are effective for real-time emotion recognition in educational contexts.
• This research offers valuable insights for emotion recognition in student learning, particularly relevant during the COVID-19 pandemic.