A Tale of Two Diverse Qualtrics Samples: Information for Online Survey Researchers

There is often a lack of transparency in research using online panels related to recruitment methods and sample derivation. The purpose of this study was to describe the recruitment and participation of respondents from two disparate surveys derived from the same online research panel using quota sampling.

A commercial survey sampling and administration company, Qualtrics, was contracted to recruit participants and implement two Internet-based surveys. The first survey targeted adults aged 50 to 75 years and used sampling quotas to obtain diversity with respect to household income and race/ethnicity. The second focused on women aged 18 to 49 years and utilized quota sampling to achieve a geographically balanced sample.

A racially and economically diverse sample of older adults (n = 419) and a geographically diverse sample of younger women (n = 530) were acquired relatively quickly (within 12 and 4 days, respectively). With exception of the highest income level, quotas were implemented as requested. Recruitment of older adults took longer (vs. younger female adults). Although survey completion rates were reasonable in both studies, there were inconsistencies in the proportion of incomplete survey responses and quality fails.

Cancer researchers, and researchers in general, should consider ways to leverage the use of online panels for future studies. To optimize novel and innovative strategies, researchers should proactively ask questions about panels and carefully consider the strengths and drawbacks of online survey features including quota sampling and forced response.

Results provide practical insights for cancer researchers developing future online surveys and recruitment protocols.See all articles in this