Deep-Learning-Based Stroke Screening Using Skeleton Data from Neurological Examination Videos

According to the Korea Institute for Health and Social Affairs, in 2017, the elderly, aged 65 or older, had an average of 2.7 chronic diseases per person. The concern for the medical welfare of the elderly is increasing due to a low birth rate, an aging population, and the lack of medical personnel. The demand for services that take user age, cognitive capacity, and difficulty into account is rising. As a result, there is an increased demand for smart healthcare systems that can lower hospital admissions and offer patients individualized care. This has motivated us to develop an AI system that can easily screen and manage neurological diseases through videos. As neurological diseases can be diagnosed by visual analysis to some extent, in this study, we set out to estimate the possibility of a person having a neurological disease from videos. Among neurological diseases, we focus on stroke because it is a common condition in the elderly population and results in high mortality and morbidity worldwide. The proposed method consists of three steps: (1) transforming neurological examination videos into landmark data, (2) converting the landmark data into recurrence plots, and (3) estimating the possibility of a stroke using deep neural networks. Major features, such as the hand, face, pupil, and body movements of a person are extracted from test videos taken under several neurological examination protocols using deep-learning-based landmark extractors. Sequences of these landmark data are then converted into recurrence plots, which can be interpreted as images. These images can be fed into convolutional neural networks to classify stroke using feature-fusion techniques. A case study of the application of a disease screening test to assess the capability of the proposed method is presented.