Automated Information Extraction Pipeline for Constructing ED Knowledge Graphs from Korean Clinical Problem Lists.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Knowledge Representation

Background:

• Emergency department (ED) problem lists are crucial for patient care but remain unstructured.
• Lack of structure hinders efficient data retrieval and analysis in clinical settings.

Purpose of the Study:

• To develop an information extraction pipeline for constructing knowledge graphs from ED problem lists.
• To compare the effectiveness of different normalization strategies and large language models (LLMs) for this task.

Main Methods:

• An information extraction pipeline was developed.
• Three normalization strategies and five LLMs were evaluated on 250 annotated ED problem lists.
• Performance was assessed using Exact F1 and Partial F1 scores.

Main Results:

• Llama 3.1 70B with Named Entity Disambiguation achieved optimal performance, with Exact F1 of 0.65 and Partial F1 of 0.70.
• This approach improved F1 scores by 6.56% compared to using raw text.
• A knowledge graph was constructed from 2,000 problem lists, containing 1,983 entities and 3,208 relations.

Conclusions:

• The developed pipeline offers a practical framework for large-scale ED knowledge graph construction.
• LLMs, particularly Llama 3.1 70B, show significant potential in structuring clinical data.
• This work paves the way for improved data utilization in emergency medicine.