An iterative topic model filtering framework for short and noisy user-generated data: analyzing conspiracy theories...

Area of Science:

• Social Media Analysis
• Computational Social Science
• Natural Language Processing

Background:

• Conspiracy theories are increasingly prevalent, amplified by social media and influencing public opinion.
• Analyzing online discourse, particularly on platforms like Twitter, is crucial for understanding societal trends.
• Twitter data presents challenges due to its noisy and sparse nature, requiring advanced pre-processing techniques.

Purpose of the Study:

• To introduce and evaluate a novel pre-processing approach, Iterative Filtering with Hashtag Pooling, for analyzing Twitter data.
• To investigate the dynamics of conspiracy theories on US Twitter during the final four months of 2020.
• To identify and characterize the main conspiracy-related topics and their association with major events like the US Presidential Elections and Covid-19.

Main Methods:

• Development of Iterative Filtering for Latent Dirichlet Allocation (LDA) pre-processing.
• Implementation of Hashtag Pooling as an additional data refinement step.
• Application of geo-spatial Twitter data and LDA Topic Models to monitor US public discourse.

Main Results:

• The Iterative Filtering and Hashtag Pooling approach effectively handles noisy Twitter data for topic modeling.
• During the study period (late 2020), mainstream conspiracy theories were overshadowed by US Presidential Election and Covid-19 discussions.
• The primary conspiracy theories identified were related to "Election Fraud" and the "Covid-19-hoax," often co-occurring with Trump-related keywords.

Conclusions:

• The proposed methods enhance the ability to study online discourses and identify relevant tweets.
• Conspiracy theories related to the 2020 US election and Covid-19 were significant but secondary to broader discussions on these events.
• The findings highlight the interconnectedness of political discourse, public health events, and the spread of specific conspiracy narratives online.