Delirium Identification from Nursing Reports Using Large Language Models.

Area of Science:

• Artificial Intelligence
• Clinical Informatics
• Natural Language Processing

Background:

• Delirium detection from unstructured nursing reports is challenging.
• Automated methods are needed to improve accuracy and efficiency.
• Large language models (LLMs) show promise for clinical text analysis.

Purpose of the Study:

• To evaluate LLMs for delirium detection in nursing notes.
• To compare keyword matching, prompting, and finetuning approaches.
• To identify the most effective LLM strategy for this task.

Main Methods:

• Utilized a manually labeled dataset from University Hospital Freiburg, Germany.
• Tested Llama3 and Phi3 large language models.
• Compared performance across keyword matching, prompting, and finetuning techniques.

Main Results:

• Both prompting and finetuning LLMs were effective for delirium detection.
• Finetuning the Phi3 (3.8B) model yielded the highest accuracy (90.24%).
• Phi3 finetuning also achieved the best AUROC (96.07%), outperforming other methods.

Conclusions:

• LLMs, particularly finetuned models like Phi3, are highly effective for automated delirium detection.
• Finetuning offers superior performance over prompting and keyword matching.
• This approach can enhance clinical decision-making and patient care through improved delirium identification.