Privacy-preserving large language models for structured medical information retrieval.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Artificial Intelligence in Healthcare

Background:

• Clinical information is predominantly unstructured free text, limiting quantitative analysis.
• Accessing and analyzing free-text clinical data is crucial for improving patient care and research.

Purpose of the Study:

• To develop and evaluate an open-source pipeline using a local large language model (LLM) for quantitative information extraction from clinical text.
• To assess the LLM's performance in identifying key features of decompensated liver cirrhosis.

Main Methods:

• Utilized the "Llama 2" large language model (LLM) for information extraction from 500 patient histories in the MIMIC IV dataset.
• Employed zero-shot and one-shot learning approaches with varying LLM sizes and prompt engineering strategies.
• Compared LLM predictions against ground truth established by three blinded medical experts.

Main Results:

• The LLM pipeline achieved high accuracy in detecting liver cirrhosis (100% sensitivity, 96% specificity).
• High performance was also observed for identifying ascites (95%, 95%), confusion (76%, 94%), abdominal pain (84%, 97%), and shortness of breath (87%, 97%).
• The 70 billion parameter LLM model demonstrated superior performance compared to smaller versions.

Conclusions:

• Locally deployed LLMs can effectively extract quantitative clinical information from free text with minimal hardware requirements.
• This approach enhances the accessibility of clinical data for quantitative analysis and supports clinical decision-making.