Data Quality and Study Compliance Among College Students Across 2 Recruitment Sources: Two Study Investigation

Area of Science:

• Psychology
• Research Methodology

Background:

• Participant recruitment methods can influence study data quality and retention.
• Satisficing behaviors may increase with higher survey burden and lower participant motivation, potentially reducing data accuracy.
• Attrition and poor data quality can negatively impact study power and generalizability.

Purpose of the Study:

• To investigate how college student recruitment sources affect data quality and retention.
• To examine the impact of data quality on study variable associations and internal consistency measures.
• To determine if demographic characteristics differ between participants who pass or fail attention checks.

Main Methods:

• Analysis of two prior studies involving college students (N=407 and N=528).
• Recruitment via psychology participation pool (course credit) versus general student body (payment/raffle).
• Inclusion of attention checks to assess data quality and tracking of follow-up survey completion for retention.

Main Results:

• Participants recruited from the psychology pool exhibited poorer data quality (more failed attention checks) and lower retention rates.
• Excluding participants who failed attention checks altered study variable associations, with negligible impact on internal consistency measures.
• Attention check failure was not significantly linked to most demographics but varied by race, suggesting filtering may preserve sample diversity.

Conclusions:

• Researchers must carefully select college student recruitment strategies and implement attention checks.
• Findings highlight the importance of monitoring data quality and retention across different recruitment channels.
• Consideration of recruitment source impact is crucial for ensuring robust and generalizable research findings.