Voice-Based Structured Nursing Documentation Using Automatic Speech Recognition and Large Language Models:...

Area of Science:

• Clinical Informatics
• Natural Language Processing
• Healthcare Technology

Background:

• Manual nursing documentation in Hospital Information Systems (HISs) is time-consuming and error-prone.
• Previous speech recognition faced challenges with code-switching, medical terms, and noisy environments.
• Advances in customized ASR and LLMs enable feasible speech-based, structured nursing documentation.

Purpose of the Study:

• To develop and evaluate an integrated ASR and LLM system for transforming spoken nursing input into structured DART notes.
• To assess the system's accuracy, usability, and clinical feasibility within HIS workflows.

Main Methods:

• Fine-tuned Whisper ASR model using a code-switching nursing speech corpus.
• Employed LLMs for schema-constrained DART record generation from ASR transcripts.
• Evaluated ASR accuracy (mixed error rate), DART classification (F1-scores), hallucination rates, and nurse feedback.

Main Results:

• Reduced ASR mixed error rate from 44.79% to 6.67%.
• Achieved a macroaveraged F1-score of 0.82 for DART generation, meeting noninferiority to human transcripts.
• Observed a 2.51% hallucination rate and a significant increase in documented notes; 75.8% of nurses reported reduced workload and improved completeness.

Conclusions:

• The integrated ASR and LLM system is feasible and performs well, demonstrating strong acceptance among clinical nurses.
• The system effectively reduces the manual documentation burden and enhances record completeness.
• Supports the value of ASR and LLM-assisted workflows for optimizing nursing documentation.