Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Traditional human activity recognition (HAR) relies on manually engineered features.
• Deep convolutional neural networks show promise for automated feature extraction from sensor data.
• Capturing temporal dynamics is crucial for accurate HAR, as activities involve complex motor sequences.

Purpose of the Study:

• To propose a generic deep learning framework for HAR that integrates convolutional and recurrent neural networks.
• To enable natural sensor fusion for multimodal wearable sensors.
• To automate feature extraction, eliminating the need for expert knowledge.

Main Methods:

• A hybrid deep learning framework combining convolutional and Long Short-Term Memory (LSTM) recurrent units.
• Application to multimodal wearable sensor data.
• Evaluation on two benchmark datasets, including a public HAR challenge dataset.

Main Results:

• The proposed framework achieved superior performance compared to non-recurrent deep networks, improving accuracy by up to 9% on a public challenge dataset.
• Demonstrated effectiveness in fusing multimodal sensor data for enhanced HAR.
• Identified key architectural hyperparameters influencing performance for optimization.

Conclusions:

• The developed deep framework effectively models temporal dynamics for HAR.
• It offers a flexible and high-performing solution for multimodal sensor fusion in activity recognition.
• Provides insights into hyperparameter optimization for improved HAR system design.