Nursing Activity Recognition for Automated Care Documentation in Clinical Settings

This paper introduces a concept for an assistance system that enables nursing staff in real-life clinical settings to reduce the time-consuming nursing documentation effort, e.g. from the morning routine. The overall goal is an AI-based documentation assistant that pre-fills the documentation record. An essential constraint in collecting and processing data is the use of sensors and features that preserve people's privacy and the acceptance of being observed. Therefore, the sensors selected are known body-worn acceleration sensors as well as far-infrared based thermal scans. During the training and evaluation phase, the use of an Azure Kinect is foreseen as well. A crucial intermediate step within the concept is the autonomous identification and learning of actions that form an activity that is recognised as a part of the entire routine added to the documentation. Both open data sets and self-recorded material tailored to the use case are to be used for this purpose. By checking the automatically generated documentation results after each treatment by the nursing staff, additional training material is to be constantly generated during the application operation in order to improve the pattern recognition systems in the processing layer in the long term. The article concludes with the specific structure of the data collection and the data currently collected. In addition, three neural networks are presented that use different sensor modalities (only motion data, only thermal images or a combination of both modalities) of the recorded data to classify care activities. A 10-fold cross validation achieves an accuracy of 89.65 % for the model using motion data and thermal images.