Artificial Intelligence-Based Models for Predicting Vaccines Critical Tweets: An Experimental Study.

Area of Science:

• Computational Social Science
• Artificial Intelligence
• Public Health Informatics

Background:

• Social media platforms like Twitter are crucial for public discourse, including discussions on vaccines.
• Identifying vaccine-critical content is essential for understanding public sentiment and combating misinformation.
• AI offers potential solutions for analyzing large volumes of social media data.

Purpose of the Study:

• To assess the effectiveness of various Artificial Intelligence (AI) models in predicting vaccine-critical tweets.
• To compare the performance of deep learning models (BERT, BERTweet) against classical AI models.
• To identify the optimal AI approach for automated detection of vaccine-critical content.

Main Methods:

• Manually labeled 800 tweets as 'vaccine-critical' or 'other'.
• Trained and tested deep learning models (BERT, BERTweet) and classical AI models (Random Forest, Logistic Regression, SVM, Naïve Bayes).
• Evaluated model performance using f1 score, accuracy, precision, and recall via three-fold cross-validation.

Main Results:

• BERTweet significantly outperformed all other evaluated AI models.
• Deep learning models showed higher predictive accuracy than classical AI approaches.
• The study successfully demonstrated AI's capability in classifying vaccine-critical tweets.

Conclusions:

• BERTweet is a highly effective AI model for automatically predicting vaccine-critical tweets.
• AI-driven analysis of social media can aid in monitoring public health discussions.
• Further research can refine AI models for nuanced content analysis in public health.