Related Experiment Video
Updated: Jun 27, 2026
09:49
Methods to Explore the Influence of Top-down Visual Processes on Motor Behavior
Published on: April 16, 2014
Research on Two-Stream Networks Integrating Physiological Features and Attention Mechanisms for Motion Classification
Wentong Wang1, Changyuan Wang1, Zehui Chen1
1School of Optoelectronic Engineering, Xi'an Technological University, Xi'an 710021, China.
Sensors (Basel, Switzerland)
|June 26, 2026
Summary
This study introduces an advanced motion recognition system using multi-modal data (ECG, PPG, IMU) for visually impaired individuals. The Attention-based Two-Stream Deep Fusion Convolutional Neural Network (ATS-DFCNN) significantly improves accuracy in complex environments.
Area of Science:
- Biomedical Engineering
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Traditional motion recognition methods struggle with accuracy and robustness in complex environments for visually impaired individuals.
- Existing multi-modal data filtering techniques often lack efficiency, impacting overall system performance.
- Visually impaired individuals require reliable motion recognition for enhanced mobility and safety.
Purpose of the Study:
- To develop a robust and accurate motion recognition system for visually impaired individuals using multi-modal data.
- To improve the filtering efficiency of multi-modal data for enhanced feature extraction.
- To enhance classification accuracy for complex activities including falls.
Main Methods:
- Utilized multi-modal data: electrocardiogram (ECG), photoplethysmography (PPG), and inertial measurement units (IMU).
- Proposed an improved wavelet filtering algorithm based on Long Short-Term Memory (LSTM) for efficient data filtering.
- Developed an Attention-based Two-Stream Deep Fusion Convolutional Neural Network (ATS-DFCNN) for motion recognition, incorporating a two-stream architecture for heterogeneous feature extraction (1D-CNN for spatial features, CNN-GRU for temporal physiological stress patterns) and an attention mechanism for dynamic feature fusion.
Main Results:
- The adaptive filtering algorithm achieved an Area Under the Curve (AUC) of 0.942, significantly enhancing feature distinctiveness.
- The ATS-DFCNN model demonstrated an average recognition accuracy of 92.2% across five activity categories (walking, standing, climbing stairs, descending stairs, falling).
- Achieved a 4.8% performance increase over single IMU modal classification and improved fall detection accuracy by reducing false alarms using physiological feedback.
Conclusions:
- The proposed ATS-DFCNN method offers a significant advancement in motion recognition for visually impaired individuals, particularly in complex scenarios.
- Multi-modal data fusion, coupled with an attention mechanism, effectively enhances recognition accuracy and robustness.
- The system provides reliable technical support for intelligent walking-aid systems, improving safety and mobility.
