Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation

Area of Science:

• Public Health
• Health Services Research
• Survey Methodology

Background:

• Population-based coverage indicators are crucial for monitoring global health progress and evaluating health programs.
• Validating these indicators against a "gold standard" is essential for ensuring data accuracy.
• The Improving Coverage Measurement (ICM) Core Group has developed a standardized approach for such validation studies.

Purpose of the Study:

• To describe a standard methodology for validating health intervention coverage indicators measured in household surveys.
• To provide guidance on designing and conducting indicator validation studies.

Main Methods:

• The study design involves comparing "gold standard" measures of health status and intervention receipt with respondent's survey answers.
• The "gold standard" utilizes external data sources, such as direct clinical observation or clinician assessment, to verify need and receipt of interventions.
• Follow-up interviews use standard and alternative survey questions to compare against existing household survey questions.

Main Results:

• Validation studies must report participation rates and reasons for non-participation at all stages.
• Key metrics include individual validity (sensitivity, specificity, AUC) and population-level validity (inflation factor), alongside missing data rates.
• Analysis should include associations between interviewer/participant characteristics and validity measures.

Conclusions:

• The described methods enable validation of respondent-reported coverage against objective measures, informing the selection of survey indicators.
• These validation approaches should be considered alongside other testing methods like cognitive interviewing and reliability testing.
• Public health researchers are encouraged to evaluate existing indicator validity and conduct validation studies to address evidence gaps.