Information Extraction and Summarization for Neurovascular Consultations with GPT-4o: A Clinical Case Study

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Decision Support

Background:

• Efficient review of extensive neurovascular patient records is crucial in outpatient settings.
• Large language models (LLMs) show potential for automating clinical information extraction but require validation.
• Time constraints necessitate reliable tools for summarizing key clinical data.

Purpose of the Study:

• To assess OpenAI's GPT-4o for generating structured summaries for neurovascular consultation preparation.
• To evaluate the efficiency gains from automating critical data extraction using GPT-4o.
• To validate the clinical reliability of LLM-generated summaries in a neurovascular context.

Main Methods:

• A prospective study involving 70 patients at a tertiary care hospital.
• GPT-4o generated structured summaries using a predefined template.
• Clinician assessment of accuracy and completeness using precision, recall, specificity, and accuracy metrics.

Main Results:

• GPT-4o demonstrated high accuracy (≥0.96) in extracting neurovascular patient data.
• Extraction consistency varied for time-stable (e.g., aneurysm location) versus dynamic data (e.g., medication lists).
• Imaging summaries were clinically useful, though effectiveness decreased with multiple prior studies; rare misattributions occurred.

Conclusions:

• GPT-4o effectively structured neurovascular patient data with high accuracy and minimal errors.
• Reliable source citation facilitated verification of LLM-generated summaries.
• Integration of LLM summaries into neurovascular consultations is supported, with future work on temporal data and privacy.