Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using...

Area of Science:

• Computer Science
• Biomedical Engineering
• Machine Learning

Background:

• Supervised training of human activity recognition (HAR) systems using body-worn inertial measurement units (IMUs) is limited by small labeled datasets.
• Cross-modality transfer approaches, like IMUTube, generate virtual IMU data from videos to augment training data.

Purpose of the Study:

• To demonstrate the use of large-scale virtual IMU datasets for training significantly more complex HAR systems.
• To develop and evaluate a novel model architecture capable of robustly training a higher number of parameters for HAR.

Main Methods:

• Utilized IMUTube to generate approximately 41 hours of virtual IMU data from YouTube exercise videos.
• Developed a new HAR model architecture with components specifically designed for IMU data, increasing trainable parameters by 1100x compared to state-of-the-art.
• Trained the model on virtual IMU data and calibrated it with only 36 minutes of real IMU data.

Main Results:

• The proposed model architecture achieved substantial performance improvements on classifying 13 dumbbell exercises.
• Demonstrated a 20% absolute F1 score improvement compared to state-of-the-art convolutional models in HAR.
• Successfully evaluated the model trained on virtual data using a real IMU dataset.

Conclusions:

• Large-scale virtual IMU datasets can effectively train highly complex HAR systems.
• The novel model architecture offers a significant advancement in HAR performance, particularly for analyzing complex physical activities.
• This approach overcomes data limitations in supervised HAR, paving the way for more sophisticated activity recognition systems.