Developing an Automatic System for Classifying Chatter About Health Services on Twitter: Case Study for Medicaid.

Area of Science:

• Health Informatics
• Computational Linguistics
• Social Media Analytics

Background:

• Social media platforms offer rich, real-time data for assessing consumer perceptions of health services.
• Analyzing this vast and diverse social media data presents significant challenges.

Purpose of the Study:

• To develop and evaluate an automated system for characterizing user-posted Twitter data related to health services.
• To utilize natural language processing and machine learning techniques for this characterization, using Medicaid as a case study.

Main Methods:

• Collected Twitter data using keywords and agency-specific handles.
• Manually labeled a sample of tweets and trained various supervised learning algorithms.
• Evaluated classifiers including support vector machine, random forest, and BERT, applying the best-performing model for analysis.

Main Results:

• A BERT-based classifier achieved the highest accuracy (81.7% and 80.7%) and F1 scores for consumer feedback.
• The BERT classifier outperformed other models like random forest and neural networks.
• Postclassification analysis showed differing distributions of tweet categories across corpora.

Conclusions:

• The proposed system offers a feasible solution for automatically categorizing health service-related social media data.
• This approach can be generalized to other health service programs beyond Medicaid.
• The annotated data and methods are available for future research.