Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study.

Area of Science:

• Computational linguistics
• Machine learning applications in mental health
• Natural Language Processing (NLP) for social media analysis

Background:

• Mental illness is a global concern, impacting many lives.
• Online mental health forums offer support and generate valuable data.
• Machine learning can analyze this data to predict mental health states.

Purpose of the Study:

• Benchmark text feature representation methods for social media posts.
• Compare downstream use with automated machine learning (AutoML) tools.
• Assess methods' ability to prioritize posts for moderator attention, specifically for self-harm contexts.

Main Methods:

• Utilized 1588 labeled posts from the CLPsych 2017 shared task (Reachout.com).
• Employed lexicon-based tools (e.g., LIWC, Empath) and pretrained neural networks (e.g., DeepMoji, USE, GPT-1) for post representation.
• Used AutoML tools (e.g., Topt, Auto-Sklearn) for classifier generation.

Main Results:

• The top system, using Generative Pretrained Transformer-1 (GPT-1) features fine-tuned on 150,000 unlabeled posts, achieved a state-of-the-art macroaveraged F1 score of 0.572.
• Performance was achieved without additional metadata or prior post information.
• Error analysis indicated a tendency to miss expressions of hopelessness.

Conclusions:

• Transfer learning is effective for risk prediction with limited labeled data.
• Fine-tuning pretrained language models enhances prediction accuracy when ample unlabeled text is available.
• This approach offers a promising avenue for proactive mental health support through online data analysis.