Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments

Area of Science:

• Behavioral research
• Internet-based data collection
• Cognitive and perceptual science

Background:

• The internet enables large-scale behavioral research, but concerns exist regarding data quality from self-selected web samples.
• Data quality is a particular concern for timed or complex cognitive and perceptual measures.
• Uncompensated, anonymous participants may exhibit reduced focus, impacting data interpretability.

Purpose of the Study:

• To investigate data quality in cognitive and perceptual experiments using self-selected web samples.
• To compare performance metrics between web-based and traditional samples.
• To determine if internet-based data collection compromises scientific rigor.

Main Methods:

• Conducted a range of cognitive and perceptual tests.
• Utilized uncompensated, anonymous, unsupervised, self-selected participants.
• Compared results from web samples to traditionally recruited/lab-tested samples.

Main Results:

• No systematic differences were found in mean performance between web and traditional samples.
• Performance variance did not differ significantly across sample types.
• Internal reliability of measures was consistent between self-selected web and traditional samples.

Conclusions:

• Self-selected web samples can yield high-quality data for cognitive and perceptual research.
• Concerns about reduced data quality are not systematically supported for internet-based studies.
• Internet-based data collection offers a viable method for robust behavioral research without compromising data integrity.