Extracting Symptoms of Complex Conditions From Online Discourse (Subreddit to Symptomatology): Lexicon-Based Approach

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Health Informatics

Background:

• Online health discourse is a valuable source for understanding patient experiences with complex medical conditions.
• Existing methods struggle to extract subtle, disease-specific symptoms from patient-generated online content.
• There's a need for tools that can identify patient-reported symptoms from social media for conditions like Polycystic Ovary Syndrome (PCOS).

Purpose of the Study:

• To develop and evaluate a method for extracting patient-reported, disease-specific symptoms from social media data.
• To characterize the prevalence and patterns of symptoms associated with PCOS as reported by patients.
• To create a comprehensive symptom list that reflects the lived experiences of individuals with PCOS.

Main Methods:

• Proposed a lexicon-based symptom extraction (LSE) method, initially using a large language model to build a symptom lexicon.
• Evaluated lexicon extraction effectiveness against human annotation using the Jaccard index.
• Utilized BioBERT embeddings with k-means clustering for symptom identification and normalization to refine the comprehensive symptom list.

Main Results:

• The LSE method significantly outperformed baseline methods, achieving a mean F1-score of 86.10 on a PCOS subreddit dataset.
• Generated a comprehensive list of 64 PCOS symptoms, covering those reported across multiple eHealth forums.
• Identified 28 emerging symptoms and 8 self-reported comorbidities associated with PCOS.

Conclusions:

• The developed LSE method effectively extracts patient-reported, disease-specific symptoms from online discourse.
• The comprehensive PCOS symptom list can reduce uncertainty for patients and healthcare providers.
• Analysis of PCOS symptomatology offers valuable insights for public health research and patient care.