COVID-Twitter-BERT: A natural language processing model to analyse COVID-19 content on Twitter

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• Transformer-based models have advanced NLP tasks.
• COVID-19 pandemic generated vast amounts of social media data.
• Domain-specific models are crucial for specialized NLP applications.

Purpose of the Study:

• Introduce COVID-Twitter-BERT (CT-BERT), a transformer model pre-trained on COVID-19 Twitter data.
• Evaluate CT-BERT's performance on various classification tasks.
• Compare CT-BERT against its base model, BERT-LARGE.

Main Methods:

• Pre-trained CT-BERT on a large corpus of COVID-19 related Twitter messages.
• Evaluated CT-BERT on five diverse classification datasets.
• Compared performance metrics against BERT-LARGE.

Main Results:

• CT-BERT demonstrated a 10-30% performance improvement over BERT-LARGE across all datasets.
• The most significant gains were observed on datasets within the target domain (COVID-19 social media content).
• Detailed performance metrics confirmed CT-BERT's superior classification capabilities.

Conclusions:

• CT-BERT offers improved performance for COVID-19 related NLP tasks, particularly on social media data.
• Domain-specific pre-training enhances model effectiveness for specialized content.
• CT-BERT has implications for public sentiment analysis and information dissemination chatbots.