Large Language Models for Efficient Medical Information Extraction

Area of Science:

• Clinical informatics
• Natural Language Processing
• Artificial Intelligence in Healthcare

Background:

• Extracting insights from unstructured clinical narrative reports is vital for efficient patient care.
• Manual review of clinical notes is time-consuming and prone to error.
• Large Language Models (LLMs) offer potential for automating medical information extraction.

Purpose of the Study:

• To evaluate the performance of ChatGPT, a Large Language Model (LLM), in extracting key clinical information from unstructured History and Physical (H&P) Notes.
• To compare ChatGPT's extraction capabilities against manual reviewers for specific health conditions.
• To identify areas where LLM performance can be improved for clinical data extraction.

Main Methods:

• Utilized ChatGPT to process a diverse sample of H&P Notes.
• Focused on extracting information related to four key conditions: family history of heart disease, depression, heavy smoking, and cancer.
• Compared ChatGPT's performance metrics (sensitivity and specificity) with those of manual reviewers.

Main Results:

• ChatGPT demonstrated high sensitivity for identifying depression and heavy smoking.
• ChatGPT showed high specificity in detecting cancer.
• Areas for improvement were identified, particularly in extracting nuanced semantic information for family history of heart disease and cancer.

Conclusions:

• ChatGPT exhibits significant potential for advancing medical information extraction from clinical narratives.
• LLMs like ChatGPT can assist healthcare professionals in more efficient patient data analysis.
• Further development is warranted to enhance LLM accuracy for complex clinical data, such as detailed family histories.