Induced lexico-syntactic patterns improve information extraction from online medical forums

Area of Science:

• Natural Language Processing
• Medical Informatics
• Computational Linguistics

Background:

• Limited tools exist for identifying medical entities in patient-authored text (PAT).
• Existing tools often fail to identify specific entity types or perform poorly on PAT.
• Identifying symptoms, conditions (SCs), and drugs/treatments (DTs) in PAT is crucial for understanding treatment efficacy and side effects.

Purpose of the Study:

• To develop a reliable method for extracting SCs and DTs from PAT.
• To learn lexico-syntactic patterns from annotated data to identify these medical entities.
• To enable the exploration of efficacy and side effects of various treatments mentioned in PAT.

Main Methods:

• Utilized seed dictionaries of SC and DT terms.
• Labeled discussion forums from MedHelp using these dictionaries.
• Iteratively induced lexico-syntactic patterns to extract new SC and DT terms.

Main Results:

• Successfully extracted previously unknown terms like 'LADA', 'stabbing pain', and 'cinnamon pills'.
• Achieved F1 scores of 58-70% for DT terms and 66-76% for SC terms on MedHelp forums.
• Demonstrated improved performance compared to MetaMap, OBA, a CRF classifier, and a prior pattern learning approach.

Conclusions:

• The developed entity extractor using lexico-syntactic patterns is effective for identifying specific entity types in PAT.
• This work represents the first known effort to extract SC and DT entities specifically from PAT.
• The system effectively learns and extracts informal medical terms commonly found in PAT but absent from standard dictionaries.