Validating open-source data on fatal police shootings against self-reports from a national sample of police agencies

Because of limitations to government data on police-related violence, researchers commonly use open-source data as the best approximation for studying the prevalence, causes, and prevention of police killings and other police-related violence in the United States. However, the comprehensiveness and accuracy of these open sources are not well known.

We compared fatal police shootings in three commonly used open sources to self-reports from a national sample of 573 U.S. police agencies from 2015 to 2019. Using ANOVA and regression methods, we assessed patterns of agreement and discrepancy by open source, year, and agency characteristics. We also examined media reports to assess factors contributing to overcounts in open sources.

Annual open-source counts were higher or lower than self-reports for 5%-9% of agencies depending on year and open source. Discrepancies varied between open sources but not consistently across years. Discrepancies were more likely and greater in magnitude for large agencies and state police, with less consistent evidence of regional variation. Overcounts in open sources appear linked to incidents involving multiple police agencies, multiple shooters, officer deaths, unclear causes of death, and agency misidentification.

Open-source data on fatal police shootings are largely accurate but should be used cautiously, particularly in agency-level analyses. Coordinated efforts by police agencies, open-source compilers, and other researchers could potentially improve the accuracy of data on fatal police shootings.