Related Experiment Video
Updated: Jun 3, 2026
An Objective and Child-friendly Assessment of Arm Function by Using a 3-D Sensor
Published on: February 12, 2018
A comprehensive deep learning model for motor phenotypes of Parkinson's disease using three-dimensional kinect V2
Yun-Ru Lai1, Chih-Cheng Huang2, Chia-Yi Lien3
1Departments of Neurology, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan; Hyperbaric Oxygen Therapy Center, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan.
Background:
Gait impairments are common for Parkinson's disease (PD). With the development of artificial intelligence (AI) technology and three-dimensional Kinect V2 Detectors, it is possible to enable more accurate characterization of gait impairment. We develop a comprehensive prediction model by combining skeleton gait energy image with relative distance and angle for PD motor phenotypes.
Research Question:
Does the hybrid convolutional neural network-long short-term memory (CNN-LSTM) deep learning model improve diagnostic accuracy and outperform CNN or LSTM models in diagnosing different motor phenotypes of PD?
Method:
We implemented and compared three deep learning architectures-CNN, LSTM, and a hybrid CNN-LSTM model. To mitigate class imbalance and enhance classification accuracy, the Synthetic Minority Oversampling Technique was applied. Feature relevance was determined using Random Forest (RF) and SHapley Additive exPlanations (SHAP), facilitating the identification of key predictors. Participants were stratified into three groups-healthy controls, non-postural instability, and gait disturbance (non-PIGD), and PIGD-based on mean scores from selected items of the Unified Parkinson's Disease Rating Scale.
Results:
The CNN-LSTM model demonstrated the highest predictive performance for PIGD classification during straight and turning walking in the off-medication state (AUC = 0.94 for both), followed by the CNN (AUC = 0.85 and 0.88) and LSTM models (AUC = 0.81 and 0.72). Moreover, the CNN-LSTM model achieved the highest classification accuracy across both on- and off-medication conditions. Using the DeLong test, we compared ROC curves of the CNN, LSTM, and hybrid CNN-LSTM models for PIGD classification across straight and turning walking tasks under both on- and off-medication conditions. The hybrid CNN-LSTM model consistently achieved significantly higher AUCs than the CNN and LSTM models in all settings.
Conclusion:
Our study demonstrated that using a hybrid CNN-LSTM deep learning model in combination with RF and/or SHAP-based feature analysis, can achieve high classification performance.
