Online versus offline: The Web as a medium for response time data collection

Area of Science:

• Cognitive Psychology
• Human-Computer Interaction
• Research Methodology

Background:

• The internet enables convenient data collection for psychological studies.
• Web programming languages facilitate complex online experiments.
• Concerns exist regarding data noise from diverse participant setups, particularly for sensitive measures like response times (RTs).

Purpose of the Study:

• To compare laboratory-based data collection with web-based methods for psychological experiments.
• To investigate the impact of hardware and software diversity on the accuracy of online response time (RT) data.
• To assess the suitability of web-based experiments for collecting reliable RT data.

Main Methods:

• A visual search task with varying set sizes was conducted in both laboratory and web-based settings.
• Response times (RTs) were collected and analyzed for differences in distribution parameters.
• An additional online sample was used to examine the effects of hardware/software diversity on RT accuracy.

Main Results:

• While the mean response times (RTs) differed between lab and web settings, other distribution parameters remained consistent.
• The setting's effect on RTs was independent of task difficulty (set size).
• Hardware and software diversity in online data collection can introduce noise, but this is partly mitigated by larger sample sizes and trial numbers.

Conclusions:

• Web-based experiments provide an acceptable and comparable source of response time (RT) data to traditional laboratory setups.
• Despite potential noise from diverse online environments, web experiments are a feasible method for psychological data collection.
• Increased sample size and trial count can effectively compensate for variability in online data collection.