Online interventions for reducing hate speech and cyberhate: A systematic review

The unique feature of the Internet is that individual negative attitudes toward minoritized and racialized groups and more extreme, hateful ideologies can find their way onto specific platforms and instantly connect people sharing similar prejudices. The enormous frequency of hate speech/cyberhate within online environments creates a sense of normalcy about hatred and the potential for acts of intergroup violence or political radicalization. While there is some evidence of effective interventions to counter hate speech through television, radio, youth conferences, and text messaging campaigns, interventions for online hate speech have only recently emerged.

This review aimed to assess the effects of online interventions to reduce online hate speech/cyberhate.

We systematically searched 2 database aggregators, 36 individual databases, 6 individual journals, and 34 websites, as well as bibliographies of published reviews of related literature, and scrutiny of annotated bibliographies of related literature.

We included randomized and rigorous quasi-experimental studies of online hate speech/cyberhate interventions that measured the creation and/or consumption of hateful content online and included a control group. Eligible populations included youth (10-17 years) and adult (18+ years) participants of any racial/ethnic background, religious affiliation, gender identity, sexual orientation, nationality, or citizenship status.

The systematic search covered January 1, 1990 to December 31, 2020, with searches conducted between August 19, 2020 and December 31, 2020, and supplementary searches undertaken between March 17 and 24, 2022. We coded characteristics of the intervention, sample, outcomes, and research methods. We extracted quantitative findings in the form of a standardized mean difference effect size. We computed a meta-analysis on two independent effect sizes.

Two studies were included in the meta-analysis, one of which had three treatment arms. For the purposes of the meta-analysis we chose the treatment arm from the Álvarez-Benjumea and Winter (2018) study that most closely aligned with the treatment condition in the Bodine-Baron et al. (2020) study. However, we also present additional single effect sizes for the other treatment arms from the Álvarez-Benjumea and Winter (2018) study. Both studies evaluated the effectiveness of an online intervention for reducing online hate speech/cyberhate. The Bodine-Baron et al. (2020) study had a sample size of 1570 subjects, while the Álvarez-Benjumea and Winter (2018) study had a sample size of 1469 tweets (nested in 180 subjects). The mean effect was small (g = -0.134, 95% confidence interval [-0.321, -0.054]). Each study was assessed for risk of bias on the following domains: randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported results. Both studies were rated as "low risk" on the randomization process, deviations from intended interventions, and measurement of the outcome domains. We assessed the Bodine-Baron et al. (2020) study as "some" risk of bias regarding missing outcome data and "high risk" for selective outcome reporting bias. The Álvarez-Benjumea and Winter (2018) study was rated as "some concern" for the selective outcome reporting bias domain.

The evidence is insufficient to determine the effectiveness of online hate speech/cyberhate interventions for reducing the creation and/or consumption of hateful content online. Gaps in the evaluation literature include the lack of experimental (random assignment) and quasi-experimental evaluations of online hate speech/cyberhate interventions, addressing the creation and/or consumption of hate speech as opposed to the accuracy of detection/classification software, and assessing heterogeneity among subjects by including both extremist and non-extremist individuals in future intervention studies. We provide suggestions for how future research on online hate speech/cyberhate interventions can fill these gaps moving forward.