The Impact of Attentiveness Interventions on Survey Data

Area of Science:

• Social and behavioral sciences
• Survey methodology
• Data quality research

Background:

• Survey data quality is crucial for social and behavioral science research.
• Common method variance (CMV) and insufficient effort responding (IER) are key threats to data integrity.
• Limited research exists on how interventions targeting IER impact CMV.

Purpose of the Study:

• To investigate the effect of attentiveness interventions on both IER and CMV in same-source survey data.
• To determine if procedural or statistical techniques to mitigate IER influence CMV.
• To assess the impact of a Factual Manipulation Check (FMC) on survey data quality.

Main Methods:

• An experiment was conducted both in-laboratory and out-of-laboratory settings.
• Attentiveness interventions, including a Factual Manipulation Check (FMC), were implemented.
• Respondent attentiveness was measured through video engagement and mouse movement tracking, alongside standard IER metrics.

Main Results:

• Attentiveness interventions showed some impact on participant engagement levels.
• Despite influencing attentiveness, these interventions did not lead to significant differences in CMV.
• No clear relationship was found between IER mitigation strategies and CMV levels.

Conclusions:

• Interventions aimed at improving respondent attentiveness may not effectively reduce common method variance.
• Researchers should consider the nuanced effects of data quality interventions on different error types.
• Further investigation is needed to understand the interplay between IER and CMV mitigation techniques.