Evaluating the vaccination coverage: validity of household-hold vaccination booklet and caregiver's recall

We compared results from household data sources to medical record sources by using data from a vaccination coverage survey.

Vaccination coverage (VC) was calculated based on parental recall, household vaccination booklet, and Zhejiang provincial immunization information system (ZJIIS). We evaluated the accuracy of VC based on household sources (vaccination booklet and recall) assuming the medical record was accurate. Concordance, sensitivity, specificity, positive predictive value, and negative predictive value were estimated as well as the Kappa statistic was also used to evaluate the agreement between data sources.

Among the 1,800 children identified in the household survey, all were registered in ZJIIS. VC estimated using the vaccination booklet alone was substantially lower than that based on medical records (net bias 3.4-16.7% in different age groups). VC based on parental recall ranged from 2.5% below (among children aged 1 year) to 16.7% points above (among children aged 6 years) than those based on medical records. Concordance was lowest for card estimates (32.5-45.5%). Sensitivity was <60% for all household sources, except for recall source. Specificity was lowest for recall estimates (14.5-42.6%). Positive predictive value was >75%, while negative predictive value was <50%, for all household sources. Kappa statistics generally indicated poor agreement between household and medical record sources.

Household-retained vaccination booklets and parental recall were insufficient sources for evaluating the VC. Our findings emphasized the importance of taking interventions to make the vaccination booklet more consistent with the records from medical resource.