A genetic algorithm enabled ensemble for unsupervised medical term extraction from clinical letters.

Area of Science:

• Natural Language Processing
• Medical Informatics
• Machine Learning

Background:

• Clinical letters remain a primary communication method between practitioners, despite the rise of electronic health records.
• Unstructured clinical text contains valuable longitudinal patient data.
• Automated extraction of medical terms from these letters is crucial for data analysis.

Purpose of the Study:

• To develop and evaluate unsupervised methods for automatic extraction of single and multi-word medical terms from clinical letters.
• To integrate multiple extraction techniques using a genetic algorithm for optimal performance.
• To assess the effectiveness of the ensemble method against domain expert annotations.

Main Methods:

• Sequential pattern mining (PrefixSpan)
• Frequency linguistic based C-Value
• Keyphrase extraction from co-occurrence graphs (TextRank)
• Genetic algorithm for parameter optimization of a linear ensemble

Main Results:

• The ensemble method achieved an average F-measure of 65.65% for complex medical terms.
• Including single-word terms, the ensemble achieved an F-measure of 72.47%.
• The ensemble performance was markedly better than individual term extraction techniques.

Conclusions:

• An ensemble of unsupervised methods effectively extracts medical terms from unstructured clinical letters.
• The proposed genetic algorithm successfully optimizes the integration of diverse extraction approaches.
• This approach offers a robust solution for leveraging valuable data within clinical correspondence.