Monitoring stance towards vaccination in twitter messages.

Area of Science:

• Computational Linguistics
• Public Health Informatics
• Social Media Analysis

Background:

• Automated stance detection is crucial for monitoring public opinion on vaccination.
• Current sentiment analysis tools perform poorly in identifying negative vaccine stances.
• Monitoring social media provides insights into vaccine hesitancy.

Purpose of the Study:

• To develop and evaluate a system for automatically classifying stance towards vaccination on Twitter, specifically focusing on negative stances.
• To improve the detection of negative vaccine sentiment compared to existing methods.
• To provide actionable insights into public vaccine hesitancy.

Main Methods:

• Annotation of Dutch Twitter messages mentioning vaccination-related keywords for stance and feeling.
• Training and testing various machine learning models using annotated data.
• Comparison of different setups based on dataset size, labeling reliability, number of categories, and classification algorithms.

Main Results:

• Support Vector Machines (SVM) trained on combined labeled data with fine-grained labeling achieved the best performance (F1-score: 0.36, AUC: 0.66).
• The developed system significantly outperformed standard sentiment analysis (F1-score: 0.25, AUC: 0.57).
• System recall for negative tweets was optimized to 0.60 with minimal precision loss.

Conclusions:

• Automated stance prediction for vaccination is challenging but achievable.
• The developed model demonstrates sufficient recall to reduce manual review efforts for negative vaccine-related tweets.
• Future improvements require larger datasets and human-in-the-loop feedback for optimal performance.