Leveraging pretrained language models for seizure frequency extraction from epilepsy evaluation reports

Area of Science:

• Medical Informatics
• Natural Language Processing
• Epilepsy Research

Background:

• Seizure frequency is critical for evaluating epilepsy treatment efficacy and patient safety.
• Clinical narratives often contain unstructured seizure frequency details, posing a challenge for data extraction.

Purpose of the Study:

• To develop and evaluate an approach for extracting structured seizure frequency information from unstructured clinical text.
• To compare the performance of BERT-based and generative large language models in this extraction task.

Main Methods:

• Fine-tuning BERT-based models (bert-large-cased, biobert-large-cased, Bio_ClinicalBERT) and generative models (GPT-4, GPT-3.5 Turbo, Llama-2-70b-hf).
• Two tasks were investigated: extracting seizure frequency phrases and extracting seizure frequency attributes.
• Integrating results from both tasks to create a final structured output.

Main Results:

• GPT-4 demonstrated superior performance across all evaluated tasks.
• GPT-4 achieved F1-scores of 85.79% for frequency phrase extraction and 91.84% for attribute extraction.
• The final integrated structured output using GPT-4 achieved an F1-score of 85.82%.

Conclusions:

• Fine-tuned generative large language models show significant potential for extracting structured data from limited clinical text.
• This approach can improve the utilization of clinical narratives for epilepsy patient management and research.