Responses to a COVID-19 vaccination intervention: Qualitative analysis of 17K unsolicited SMS replies

The development of effective interventions for COVID-19 vaccination has proven challenging given the unique and evolving determinants of that behavior. A tailored intervention to drive vaccination uptake through machine learning-enabled personalization of behavior change messages unexpectedly yielded a high volume of real-time short message service (SMS) feedback from recipients. A qualitative analysis of those replies contributes to a better understanding of the barriers to COVID-19 vaccination and demographic variations in determinants, supporting design improvements for vaccination interventions.

The purpose of this study was to examine unsolicited replies to a text message intervention for COVID-19 vaccination to understand the types of barriers experienced and any relationships between recipient demographics, intervention content, and reply type.

We categorized SMS replies into 22 overall themes. Interrater agreement was very good (all κ_pooled > 0.62). Chi-square analyses were used to understand demographic variations in reply types and which messaging types were most related to reply types.

In total, 10,948 people receiving intervention text messages sent 17,090 replies. Most frequent reply types were "already vaccinated" (31.1%), attempts to unsubscribe (25.4%), and "will not get vaccinated" (12.7%). Within "already vaccinated" and "will not get vaccinated" replies, significant differences were observed in the demographics of those replying against expected base rates, all p > .001. Of those stating they would not vaccinate, 34% of the replies involved mis-/disinformation, suggesting that a determinant of vaccination involves nonvalidated COVID-19 beliefs.

Insights from unsolicited replies can enhance our ability to identify appropriate intervention techniques to influence COVID-19 vaccination behaviors. (PsycInfo Database Record (c) 2023 APA, all rights reserved).