Disentangling Selection Into Mode From Mode Effects

Area of Science:

• Gerontology
• Survey Methodology
• Sociology

Background:

• The National Social Life, Health, and Aging Project (NSHAP) collects crucial data on aging.
• Data collection modes can influence survey responses.
• Understanding these influences is vital for accurate research.

Purpose of the Study:

• To assess the impact of remote versus in-person data collection on NSHAP Round 4 responses.
• To guide appropriate methods for addressing mode differences in analyses.

Main Methods:

• A randomized experiment compared remote and in-person interviews for a subset of NSHAP respondents.
• Response rates and key outcomes were compared between experimental groups.
• All Round 4 respondents were analyzed based on their final data collection mode.

Main Results:

• No significant differences in physical/mental health or simple recall were found between remote and in-person modes in the randomized experiment.
• In-person interviews showed slightly lower cognitive function scores on specific tasks.
• Remote respondents reported fewer social network members.
• Comparing all respondents by final mode yielded substantially different, potentially misleading, results.

Conclusions:

• Data collection mode did not significantly affect self-reported health in NSHAP.
• Mode moderately impacted other variables, such as social network size and certain cognitive measures.
• Naive comparisons of all respondents by mode are inappropriate for adjusting data and can yield misleading findings.