Measuring vaccination coverage and concerns of vaccine holdouts from web search logs

To design effective vaccine policies, policymakers need detailed data about who has been vaccinated, who is holding out, and why. However, existing data in the US are insufficient: reported vaccination rates are often delayed or not granular enough, and surveys of vaccine hesitancy are limited by high-level questions and self-report biases. Here we show how search engine logs and machine learning can help to fill these gaps, using anonymized Bing data from February to August 2021. First, we develop a vaccine intent classifier that accurately detects when a user is seeking the COVID-19 vaccine on Bing. Our classifier demonstrates strong agreement with CDC vaccination rates, while preceding CDC reporting by 1-2 weeks, and estimates more granular ZIP-level rates, revealing local heterogeneity in vaccine seeking. To study vaccine hesitancy, we use our classifier to identify two groups, vaccine early adopters and vaccine holdouts. We find that holdouts, compared to early adopters matched on covariates, are 67% likelier to click on untrusted news sites, and are much more concerned about vaccine requirements, development, and vaccine myths. Even within holdouts, clusters emerge with different concerns and openness to the vaccine. Finally, we explore the temporal dynamics of vaccine concerns and vaccine seeking, and find that key indicators predict when individuals convert from holding out to seeking the vaccine.