Discriminative Cut-Offs, Concurrent Criterion Validity, and Test-Retest Reliability of the Oxford Vaccine Hesitancy...

Area of Science:

• Public Health
• Psychometrics
• Vaccinology

Background:

• Validated instruments for quantifying vaccine hesitancy are crucial but often lack transportable cut-offs and temporal stability metrics.
• The Oxford Vaccine Hesitancy Scale (OVHS) was evaluated to address these limitations.

Purpose of the Study:

• To quantify the OVHS's discrimination for historical COVID-19 vaccine refusal.
• To derive and validate a single, transportable operational cut-off score across populations.
• To assess the test-retest reliability of the OVHS.

Main Methods:

• Analysis of five datasets (N=2451) from UK and US respondents using Receiver Operating Characteristic (ROC) curves.
• Application of a Youden's J plateau algorithm with a specificity floor (≥0.75) to determine a transportable cut-off.
• Assessment of test-retest reliability using Intraclass Correlation Coefficient (ICC).

Main Results:

• Area Under the Curve (AUC) values ranged from 0.760 to 0.971, indicating good discrimination.
• An operational cut-off of OVHS ≥ 35 was derived and validated, with sensitivity/specificity ranging from 0.73-0.95/0.63-0.87.
• Good to excellent test-retest reliability was observed (ICC(A,1) = 0.884).

Conclusions:

• The OVHS demonstrates robust discrimination for COVID-19 vaccine refusal and strong temporal stability.
• A single, transparent, and transportable operational cut-off (≥35) was successfully established.
• The framework for cut-off derivation may be generalizable to other hesitancy instruments.