Large language models outperform traditional structured data-based approaches in identifying immunosuppressed...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Natural Language Processing

Background:

• Identifying immunosuppressed patients is crucial for clinical care but challenging using solely structured electronic health record data.
• Unstructured clinical text contains valuable information often missed by traditional data extraction methods.

Purpose of the Study:

• To evaluate the effectiveness of large language models (LLMs) in identifying immunosuppressed patients from unstructured clinical notes.
• To compare the performance of advanced LLMs against traditional data extraction approaches.

Main Methods:

• Utilized GPT-4o to process hospital admission notes for identifying immunosuppressive conditions and medication use.
• Assessed the performance of GPT-4o and compared it with traditional methods.
• Validated the approach on an external dataset to demonstrate extensibility.

Main Results:

• GPT-4o significantly outperformed traditional methods in accurately identifying immunosuppressed patients from clinical notes.
• Cost-effective models, including GPT-4o mini and Llama 3.1, demonstrated strong performance, though not matching GPT-4o.
• The LLM-based approach proved extensible to an external dataset.

Conclusions:

• Large language models, particularly GPT-4o, offer a superior method for identifying immunosuppressed patients compared to traditional approaches.
• LLMs can effectively leverage unstructured clinical text for improved patient cohort identification.
• Accessible LLM variants provide viable, cost-effective alternatives for clinical data extraction.