Unsupervised method for automatic construction of a disease dictionary from a large free text collection

Area of Science:

• Medical Informatics
• Natural Language Processing
• Biomedical Research

Background:

• Medical language processing systems rely on concept-specific lexicons (e.g., diseases, drugs).
• Existing medical dictionaries are often incomplete due to the rapid advancement of biomedical research.
• Automated methods are needed to create comprehensive and up-to-date medical terminologies.

Purpose of the Study:

• To develop an automated, unsupervised, iterative pattern learning approach for constructing a comprehensive disease dictionary.
• To compare different ranking methods for extracting contextual patterns and disease terms.
• To enhance the accuracy of disease concept identification in medical texts.

Main Methods:

• An unsupervised, iterative pattern learning approach was employed to construct a disease dictionary.
• The method focused on extracting disease terms from randomized clinical trial (RCT) abstracts.
• Various ranking methods were evaluated for pattern and concept term extraction.

Main Results:

• The automated disease dictionary significantly improved performance in identifying disease concepts.
• Performance gains (F1 score increase of 35-88%) were observed compared to manually created terminologies.
• The approach successfully extracted disease terms from 100 manually annotated clinical abstracts.

Conclusions:

• Automated dictionary construction from RCT abstracts is effective for enhancing medical language processing.
• The developed approach offers a significant improvement over existing manually curated disease terminologies.
• This method provides a scalable solution for maintaining comprehensive and accurate medical knowledge bases.