Automatic Evaluation of Motor Rehabilitation Exercises Based on Deep Mixture Density Neural Networks

An automatic assessment system for physical telerehabilitation could reduce the time and cost of treatments. But such assessment involves stochastic uncertainties, nonlinearities, and complexities of human movement. Probabilistic models and deep structures are two categories that could, respectively, address the stochastic uncertainty and complexity of motion data. In the proposed Deep Mixture Density Network (DMDN), probabilistic models were concurrently processed along with deep neural networks. More specifically, a multi-branch convolutional layer extracted the deep features of motion data, a Long Short Term Memory (LSTM) learned its temporal dependencies, and a Gaussian Mixture Model (GMM) handled the stochastic interaction of its preceding layers in reaching a more valid assessment and improved generalization to new movements. Finally, the weighted mean of the GMM components was used as the performance score for exercises. Input data were the time series related to the joints' position and orientation extracted from the Kinect v2 sensor video. A clinical reference score for each movement was also included for training the DMDN. In addition to comparisons with the state-of-the-art algorithms, an ablation study of the various elements comprising the DMDN was performed. Three configurations of convolutional filter window transitions across input data were also investigated. Results indicate that the proposed DMDN with one-dimensional parallel window transitions outperforms the other competing strategies in the ablation study. It also achieves higher reliability in terms of a lower RMSE standard deviation against a DMDN without GMM strategy while ranking competitively according to the Spearman correlation coefficient and Root Mean Square Error.