Text classification for assisting moderators in online health communities

Area of Science:

• Health Informatics
• Computational Social Science
• Natural Language Processing

Background:

• Online health communities are vital for patient support and information exchange.
• The scale of these communities challenges moderators' ability to engage in all necessary conversations.
• Automated methods are needed to identify critical discussions requiring moderator intervention.

Purpose of the Study:

• To explore low-cost text classification methods for identifying health forum threads that require moderator attention.
• To improve the accuracy of automated systems in flagging important online health discussions.

Main Methods:

• A binary classifier was trained on online diabetes community data.
• Features included word unigrams, sentiment analysis, and thread length.
• Feature selection (chi-squared) and under-sampling were used for imbalanced data.

Main Results:

• The classification model achieved an AUC of 0.75 and an F1-score of 0.54 with sentiment features, feature selection, and balanced data.
• This represents a significant improvement over the baseline model (0.65 AUC, 0.40 F1-score).
• Qualitative error analysis revealed additional factors influencing moderator responses.

Conclusions:

• Feature selection and balanced training data enhance classification performance for online health forums.
• Findings have implications for balancing precision and recall in moderator assistance tools.
• Error analysis highlighted social, legal, and ethical considerations in addressing patient needs online.