Smallpox vaccination: comparison of self-reported and electronic vaccine records in the millennium cohort study

In December 2002, the US Government implemented policy to immunize health workers, first responders, and military personnel against smallpox in preparation for a possible bioterrorist attack. Self-reported vaccination data are commonly used in epidemiologic research and may be used to determine vaccination status in a public health emergency. To establish a measure of reliability, the agreement between self-reported smallpox vaccination and electronic vaccination records was examined using data from the Millennium Cohort Study. Descriptive measures and a kappa statistic were calculated for data from 54,066 Millennium Cohort Study participants. Multivariable modeling adjusting for potential confounders was used to investigate vaccination agreement status and health metrics, as measured by the Short Form 36-Item Health Survey for Veterans (SF-36V) and hospitalization data. Substantial agreement (kappa =0.62) was found between self-report and electronic recording of smallpox vaccination. Of all participants with an electronic record of smallpox vaccination, 90% self-reported being vaccinated; and of all participants with no electronic record of vaccination, 82% self-reported not receiving a vaccination. There was no significant difference in hospitalization experience prior to questionnaire completion between vaccinated and unvaccinated participants. While overall scores on the SF-36V suggested a healthy population, participants whose self-reported vaccination status did not match electronic records had slightly lower adjusted mean scores for some scales. These results indicate strong reliability in self-reported smallpox vaccination and also suggest that discordant reporting of smallpox vaccination is not associated with substantial differences in health among Millennium Cohort participants.