When Who Matters: Interviewer Effects and Survey Modality.

Area of Science:

• Social Sciences
• Survey Methodology
• Data Quality

Background:

• Survey modality choice is constrained by budget and external factors.
• Different survey methods have varying susceptibility to measurement error from interviewer effects.
• Interviewer effects can significantly impact the reliability of survey data.

Purpose of the Study:

• To examine variations in interviewer effects across different survey modalities (face-to-face vs. phone).
• To investigate how question type (simple vs. sensitive) influences interviewer effects in different modalities.
• To assess the implications of these findings for survey design and power calculations.

Main Methods:

• Comparison of interviewer effects in highly similar face-to-face (FTF) and telephone surveys.
• Analysis of interviewer effects based on question sensitivity.
• Leveraging existing survey data to identify modality-specific measurement errors.

Main Results:

• No significant cross-modality differences in interviewer effects for simple questions were observed.
• Interviewer effects were substantially higher for sensitive questions administered via telephone compared to FTF.
• In-person interviewers appear more effective at building rapport, mitigating effects for sensitive questions.

Conclusions:

• Telephone surveys exhibit greater interviewer effects for sensitive questions than FTF surveys.
• Findings suggest FTF data may lead to underestimation of sample size needs for phone surveys on sensitive topics.
• Survey designers must consider modality-specific interviewer effects for accurate data collection and analysis.