Extracting medication information from unstructured public health data: a demonstration on data from population-based...

Area of Science:

• Clinical Epidemiology
• Natural Language Processing
• Data Science

Background:

• Unstructured data from clinical studies is valuable but difficult to analyze manually.
• Manual data extraction is time-consuming, labor-intensive, and prone to errors.
• Automation is needed to efficiently process large volumes of unstructured clinical data.

Purpose of the Study:

• To develop and demonstrate an automation framework for extracting and processing unstructured clinical data.
• To reduce the manual effort required for data analysis in epidemiological studies.
• To improve the efficiency and accuracy of data processing for medication and reasons for use.

Main Methods:

• Utilized two natural language processing (NLP) tools for medication and reason extraction.
• Applied spell-checking and mapped medication names to generic and Anatomical Therapeutic Chemical (ATC) classifications.
• Processed reasons for medication using the Lancaster stemmer and mapped them to disease classes based on organ systems.
• Demonstrated the framework on Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) data.

Main Results:

• Successfully condensed 1266 distinct medication names into 89 ATC categories.
• Reduced 1432 distinct reasons for medication use into 65 categories using NLP.
• Automation reduced manual mapping effort by 84.4% for medications and 59.4% for reasons.
• Improved the precision of mapped results compared to manual processing.

Conclusions:

• The NLP-based automation framework effectively processes unstructured clinical data.
• The method is adaptable for less established databases and can incorporate new knowledge sources.
• Condensing features into interpretable categories enhances subsequent machine learning and data mining studies.