Data quality and response distributions in a mixed-mode survey.

Area of Science:

• Survey Methodology
• Social Statistics
• Data Collection Techniques

Background:

• Traditional interviewer-administered longitudinal surveys face cost and logistical pressures.
• Sequential mixed-mode designs offer a cost-effective alternative, starting with self-administered online surveys followed by interviewer-based follow-ups.
• The Health and Retirement Study (HRS) is a key longitudinal survey facing these pressures.

Purpose of the Study:

• To compare a sequential mixed-mode design (web-then-telephone) with a telephone-only design.
• To evaluate response quality and distributions across key HRS domains: health, finances, expectations, and family composition.
• To identify and quantify measurement differences between survey modes.

Main Methods:

• A designed experiment within the 2018 Health and Retirement Study (HRS) wave.
• Comparison of a sequential web-then-telephone mode against a telephone-only mode.
• Intent-to-treat analysis focusing on response quality metrics and domain-specific data.

Main Results:

• Sequential mixed-mode (web) showed slightly higher missing data rates and more focal responses than telephone-only.
• No significant differences in verifying/updating roster information were found between modes.
• Slightly lower asset ownership reported in web mode, but no mode effect on asset value conditional on ownership.
• More pessimistic expectations reported in web mode; minimal evidence of poorer health reporting in web mode.

Conclusions:

• Mode assignment effects on measurement exist but are generally small across most indicators.
• Remediation strategies for item-missing data and focal values are crucial for reducing mode effects.
• Sequential mixed-mode designs are viable alternatives, but require careful management of specific data quality differences.