Integrating a spoken dialogue system, nursing records, and activity data collection based on smartphones

This study describes the integration of a spoken dialogue system and nursing records on an Android smartphone application intending to help nurses reduce documentation time and improve the overall experience of a healthcare setting. The application also incorporates with collecting personal sensor data and activity labels for activity recognition.

We developed a joint model based on a bidirectional long-short term memory and conditional random fields (Bi-LSTM-CRF) to identify user intention and extract record details from user utterances. Then, we transformed unstructured data into record inputs on the smartphone application.

The joint model achieved the highest F1-score at 96.79%. Moreover, we conducted an experiment to demonstrate the proposed model's capability and feasibility in recording in realistic settings. Our preliminary evaluation results indicate that when using the dialogue-based, we could increase the percentage of documentation speed to 58.13% compared to the traditional keyboard-based.

Based on our findings, we highlight critical and promising future research directions regarding the design of the efficient spoken dialogue system and nursing records.