Enhancing Adverse Event Reporting With Clinical Language Models: Inpatient Falls | JoVE Visualize

Area of Science:

Natural Language Processing
Clinical Informatics
Patient Safety

Background:

Inpatient falls are frequently underreported, with self-reporting mechanisms missing up to 91% of incidents.
Accurate tracking of adverse events like falls is crucial for patient safety and quality improvement.
Existing methods for fall detection often rely on manual reporting, which can be burdensome and incomplete.

Purpose of the Study:

To develop and evaluate a computational method for detecting patient fall events using clinical language models.
To assess the performance of various language models, including Bidirectional Encoder Representations from Transformers (BERT) and Generative Pretrained Transformer (GPT)-4, in identifying falls from clinical notes.
To compare the effectiveness of prompt programming with standardized prompts for GPT-4 in fall detection.

Main Methods:

A retrospective observational study utilizing unstructured nursing notes from electronic health records and national patient safety reports.
Data preprocessing involved anonymization, English translation, and semantic validation of 34,480 records (January 2015 - December 2019).
Five language models, including fine-tuned BERT and GPT-4 with prompt programming, were explored, with performance measured by F1 scores and error analysis.

Main Results:

Fine-tuned BERT models achieved the highest performance, with Bio+Clinical BERT and Korean BERT both reaching an F1 score of 0.98.
GPT-4 with prompt programming demonstrated significantly improved performance (F1 score of 0.94 for Korean data, 0.85 for English data) compared to standardized prompts.
Common errors included misclassification due to fall history, homonyms (false positives), and implicit expressions or missing context (false negatives).

Conclusions:

Clinical language models offer a promising approach to computationally detect patient falls, significantly improving the identification of unreported incidents.
This method can enhance adverse event tracking accuracy and reduce the reliance on manual self-reporting by nurses.
Integrating language model-based fall detection with existing self-reporting systems can lead to more comprehensive and accurate patient safety monitoring.