A Large Language Model-Based Approach for Coding Information from Free-Text Reported in Fall Risk Surveillance...

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Patient Safety Research

Background:

• In-hospital falls are a frequent adverse event with significant patient and healthcare system costs.
• Current methods for analyzing fall reports can be time-consuming and resource-intensive.

Purpose of the Study:

• To develop and implement an automated coding system using large language models (LLMs) for in-hospital fall records.
• To extract and categorize key information such as fall location and injury status from free-text narratives.

Main Methods:

• Utilized narrative descriptions from an Italian Local Health Authority's Incident Reporting system.
• Employed OpenAI's GPT-4-turbo models via API for data extraction and classification.
• Established a gold standard using manual coding by two independent reviewers for performance evaluation.

Main Results:

• GPT-4-turbo demonstrated high performance in detecting fall location (specificity, sensitivity, accuracy ≥ 0.913) and injury status (specificity, sensitivity, accuracy ≥ 0.953).
• The models effectively processed unstructured free-text data, even without pre-optimization.

Conclusions:

• Large language models show significant potential for automating data extraction and categorization in clinical risk management.
• This approach can enhance the efficiency and accuracy of analyzing adverse event reports, improving patient safety.