Information Extraction and Summarization for Neurovascular Consultations with GPT-4o: A Clinical Case Study

In outpatient settings, extensive patient records must frequently be reviewed under time constraints, making efficient extraction and summarization of key clinical information essential. Large language models (LLMs) are potentially useful for this task but require validation for clinical reliability. This study assesses OpenAI's GPT-4o for generating structured summaries to assist in neurovascular consultation preparation, aiming to increase efficiency by automating critical data extraction.

A prospective study was conducted from May to August 2024 at a tertiary care hospital, involving a total of 70 patients. Structured summaries were generated by GPT-4o using a predefined template. Extracted data were categorized into aneurysm-specific details, imaging summaries, and patient-specific clinical factors. Accuracy and completeness were assessed by clinicians, with performance measured using precision, recall, specificity, and accuracy.

High accuracy (≥ 0.96) was measured across most categories. In aneurysm-and patient-specific data, extraction performance varied based on stability over time. Aneurysm location and other stable details were extracted consistently, while changes in aneurysm size and medication lists showed variations. In rare cases, aneurysm details were misattributed to a different aneurysm within the same patient. Imaging summaries were generally concise and clinically useful, though their effectiveness declined when summarizing multiple prior studies.

Neurovascular patient data was effectively structured by GPT-4o, demonstrating high accuracy with minimal errors. While occasional misattributions like outdated information were observed, reliable citation of sources facilitated easy verification. These findings support integrating LLM-generated summaries into neurovascular consultations, with further optimization needed for temporal data tracking and on-premise implementation to address privacy concerns.