Semantic Analysis and Topic Modelling of Web-Scrapped COVID-19 Tweet Corpora through Data Mining Methodologies

Area of Science:

• Social Sciences
• Computer Science
• Public Health

Background:

• The COVID-19 pandemic significantly impacted global social, economic, and psychological well-being.
• Twitter emerged as a key platform for public discourse and sentiment expression regarding COVID-19 and related measures.
• Understanding public psychological reactions is crucial for effective public health communication and policy.

Purpose of the Study:

• To analyze the psychological reactions and discourse of Twitter users concerning COVID-19.
• To compare the effectiveness of various machine learning models for sentiment analysis of pandemic-related tweets.
• To identify the most accurate model for classifying tweet polarity and understanding public sentiment.

Main Methods:

• Latent Dirichlet Allocation (LDA) for topic modeling of tweets.
• Bidirectional Long Short-Term Memory (BiLSTM) and other classifiers (Random Forest, SVM, Logistic Regression, Naive Bayes, Decision Tree, SGD, Voting) for sentiment polarity analysis.
• Dual dataset approach incorporating word clouds for enhanced analysis and model validation.

Main Results:

• The Bidirectional Long Short-Term Memory (BiLSTM) model demonstrated superior performance in sentiment analysis.
• BiLSTM achieved a high accuracy of 96.7% in classifying the polarity of COVID-19 related tweets.
• LDA effectively identified key topics within the public discourse on Twitter.

Conclusions:

• Machine learning, particularly BiLSTM, offers a robust method for analyzing public sentiment and psychological responses to health crises like COVID-19.
• The findings highlight the utility of social media data in understanding public perception during pandemics.
• Accurate sentiment analysis can inform public health strategies and interventions.