Multimodal Sensor Motion Intention Recognition Based on Three-Dimensional Convolutional Neural Network Algorithm.

Area of Science:

• Biomedical Engineering
• Machine Learning
• Human-Computer Interaction

Background:

• Advancements in microelectronics and computer systems drive research in motion intention recognition.
• Multimodal sensor data analysis is crucial for understanding human movement.
• Deep learning models excel in processing large datasets for complex pattern recognition.

Purpose of the Study:

• To develop a novel motion intention recognition algorithm using multimodal sensor fusion.
• To extract and fuse spatiotemporal features for enhanced movement pattern recognition.
• To determine the minimal muscle and EMG signal requirements for real-time lower limb movement recognition.

Main Methods:

• Utilized a three-dimensional convolutional neural network (3D CNN) for short-term spatiotemporal feature extraction.
• Employed Long Short-Term Memory (LSTM) networks for time-series modeling and fusion of features.
• Implemented a multimodal fusion approach combining short-term and long-term spatiotemporal features.
• Analyzed lower limb movement patterns to identify essential muscle and EMG signal characteristics.

Main Results:

• The proposed algorithm successfully fuses multimodal long-term and short-term spatiotemporal features.
• Achieved higher discrimination in recognizing movement states compared to traditional methods.
• Identified the minimum set of muscles and EMG features necessary for accurate lower limb movement recognition.
• Demonstrated the system's capability for real-time output with minimized computational cost.

Conclusions:

• The multimodal fusion algorithm offers a robust approach to motion intention recognition.
• Optimizing feature selection reduces computational load and ensures real-time performance.
• This method has significant implications for applications requiring accurate and timely human movement analysis.