Methodological rigor with internet samples: new ways to reach underrepresented populations

Area of Science:

• Social Sciences
• Internet Research Methodology
• Survey Methodology

Background:

• Internet-based research offers potential for reaching diverse populations.
• Traditional sampling methods may struggle with difficult-to-reach or underrepresented groups.
• Assessing the representativeness of online samples is crucial for valid research.

Purpose of the Study:

• To present and evaluate rigorous internet sampling methodologies.
• To compare the representativeness of internet samples with traditional random digit dialing methods.
• To determine the effectiveness of online sampling for underrepresented populations.

Main Methods:

• Development of rigorous sampling designs for internet-based data collection.
• Comparison of demographic characteristics between an internet sample (n=82) and a large Gallup Organization sample (n>1,000).
• Analysis of representativeness across age, education, income, ethnicity, and rural/urban residence.

Main Results:

• Internet sampling designs were robust and equally representative of the U.S. general population compared to random digit dialing.
• The internet sample showed greater representation in terms of lower education, lower income, and ethnic diversity.
• Both sampling methods were equally effective in representing rural and urban population distributions.

Conclusions:

• Rigorous internet sampling is a viable and effective method for surveying diverse populations.
• Online methodologies can achieve demographic representativeness comparable to, and in some aspects exceeding, traditional methods.
• Internet sampling offers a powerful tool for social science research involving underrepresented groups.