The perils of straying from protocol: sampling bias and interviewer effects

Area of Science:

• Social Sciences
• Economics
• Public Health

Background:

• Fidelity to research protocols is crucial for valid study outcomes.
• Interviewer effects can introduce bias in survey data collection.
• Previous studies have highlighted the potential for bias in survey research.

Purpose of the Study:

• To investigate interviewer effects on survey responses in a contingent valuation study.
• To assess the impact of a minor sampling protocol adaptation on data collection.
• To identify demographic differences in samples collected by different interviewers.

Main Methods:

• A contingent valuation study was conducted in an informal urban settlement in Nairobi, Kenya.
• Three trained interviewers administered the survey, with a protocol allowing "closest neighbor" contact if no one was home.
• Multinomial logistic regression and post-estimation tests were used to analyze interviewer-specific demographic differences.

Main Results:

• Significant differences were found in the demographic characteristics of samples collected by the three interviewers.
• Female interviewers were less likely to engage respondents of low socio-economic status compared to the male interviewer.
• A slight adaptation in sampling protocol led to magnified interviewer effects and inferred sampling bias.

Conclusions:

• Strict adherence to sampling protocols and consistent interviewer training are essential.
• Monitoring responses by interview is necessary to ensure data similarity and unbiased estimates.
• Interviewer choice, even minor, can introduce systematic error and compromise data integrity.