Development and Testing of a Daily Activity Recognition System for Post-Stroke Rehabilitation

Those who survive the initial incidence of a stroke experience impacts on daily function. As a part of the rehabilitation process, it is essential for clinicians to monitor patients' health status and recovery progress accurately and consistently; however, little is known about how patients function in their own homes. Therefore, the goal of this study was to develop, train, and test an algorithm within an ambient, in-home depth sensor system that can classify and quantify home activities of individuals post-stroke. We developed the Daily Activity Recognition and Assessment System (DARAS). A daily action logger was implemented with a Foresite Healthcare depth sensor. Daily activity data were collected from seventeen post-stroke participants' homes over three months. Given the extensive amount of data, only a portion of the participants' data was used for this specific analysis. An ensemble network for activity recognition and temporal localization was developed to detect and segment the clinically relevant actions from the recorded data. The ensemble network, which learns rich spatial-temporal features from both depth and skeletal joint data, fuses the prediction outputs from a customized 3D convolutional-de-convolutional network, customized region convolutional 3D network, and a proposed region hierarchical co-occurrence network. The per-frame precision and per-action precision were 0.819 and 0.838, respectively, on the test set. The outcomes from the DARAS can help clinicians to provide more personalized rehabilitation plans that benefit patients.