Motion Primitives Classification Using Deep Learning Models for Serious Game Platforms.

Area of Science:

• Computer Science
• Cultural Heritage Studies
• Human-Computer Interaction

Background:

• Serious games are increasingly used for cultural heritage (CH) applications, particularly in dance education.
• Machine learning (ML) is crucial for analyzing user interactions and providing feedback in serious games.
• Existing ML models struggle with noisy data from low-cost sensors and rich visual information.

Purpose of the Study:

• To develop an intelligent deep learning model for motion primitive classification in CH serious games.
• To enhance user progress monitoring and scoring accuracy in interactive dance applications.
• To create a robust model that handles skeleton errors and utilizes RGB data effectively.

Main Methods:

• Introduction of a novel deep learning architecture: Convolutionally Enhanced Bidirectional Long Short-Term Memory (CEBi-LSTM).
• The model integrates convolutional layers for RGB data processing with bidirectional LSTM for temporal analysis.
• Designed to be robust against skeleton errors from low-cost sensors like Kinect.

Main Results:

• The CEBi-LSTM model efficiently processes RGB information through convolutional hierarchies.
• It retains the bidirectional analysis capabilities of LSTM for sequential motion data.
• Demonstrates reduced sensitivity to skeleton errors while effectively handling detailed RGB visual information.

Conclusions:

• The CEBi-LSTM architecture offers a significant advancement in motion primitive classification for cultural heritage serious games.
• This approach enhances the intelligence and accuracy of user interactivity analysis in dance applications.
• The model provides a more reliable and detailed method for monitoring user progress in serious games utilizing visual and motion data.