Extraction of Normalized Symptom Mentions From Clinical Narratives Using Large Language Models

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Artificial Intelligence in Healthcare

Background:

• Patient symptoms are crucial for clinical decision-making but are often unstructured in electronic health records.
• Extracting symptom information from free clinical text presents significant challenges.
• Current methods for symptom extraction are limited in scope and accuracy.

Purpose of the Study:

• To evaluate the efficacy of large language models (LLMs) in extracting symptom concepts from clinical narratives.
• To compare LLM performance against traditional symptom-specific machine learning classifiers.
• To explore the potential of LLMs for improving healthcare workflows and research.

Main Methods:

• Utilized large language models (LLMs) with prompt engineering techniques, including clarifying information, few-shot examples, and chain-of-thought prompting.
• Compared LLM performance to symptom-specific machine learning classifiers trained on mapped clinical concepts.
• Evaluated performance using F1-scores for common symptom concepts.

Main Results:

• LLMs demonstrated superior performance in extracting most symptom concepts compared to machine learning classifiers.
• The LLM approach achieved higher F1-scores, indicating improved accuracy and effectiveness.
• LLMs likely leverage contextual information within clinical narratives for better symptom normalization.

Conclusions:

• Large language models offer a promising approach for unlocking valuable symptom information from unstructured clinical text.
• LLM-based symptom extraction can enhance clinical workflows and support diverse research initiatives.
• This technology has the potential to significantly advance the use of clinical narrative data in healthcare.