Validation of self-reported mpox vaccination in men who have sex with men

The Australian Government introduced a nationwide mpox MVA-BN vaccine to all eligible individuals at higher risk of mpox exposure, specifically gay, bisexual, and other men who have sex with men (GBMSM). Self-reported vaccination status is commonly used in surveillance and behavioural research, but its accuracy for mpox vaccination is unknown. We conducted a retrospective cross-sectional study of GBMSM aged ≥16 years who attended the Melbourne Sexual Health Centre (MSHC) between 7 December 2023 and 31 December 2024. Self-reported mpox vaccination status using computer-assisted self-interviewing (CASI) was compared to verified electronic health records. Sensitivity, specificity, and agreement (κ) were calculated alongside other measures of performance. Of 6050 eligible men, 5885 men self-reported their mpox vaccination status: 48.6% (n = 2859) vaccinated, 44.6% (n = 2623) unvaccinated, and 6.8% (n = 403) unsure. Self-reported mpox vaccination had a sensitivity of 96.9% (95% CI: 95.9% to 97.7%), a specificity of 65.4% (95% CI: 63.9% to 66.9%), a positive predictive value of 52.4% (95% CI: 50.6% to 54.3%), a negative predictive value of 98.2% (95% CI: 97.6% to 98.6%), and an accuracy of 74.3% (95% CI: 73.1% to 75.5%). Our results demonstrated that the majority of GBMSM correctly reported their vaccination status. While self-reported mpox vaccination demonstrates high sensitivity, moderate agreement indicates residual misclassification and incomplete capture of vaccinations in the electronic health record used for validation. This suggests that while self-report is reliable for confirming non-vaccination, it is less reliable for identifying vaccinated individuals, underscoring the importance of electronic health record verification for accurate vaccine surveillance.