Related Experiment Video
Updated: Jun 21, 2026
09:44
Methods to Test Visual Attention Online
Published on: February 19, 2015
Web-based image norming: how do object familiarity and visual complexity ratings compare when collected in-lab versus
1University of Toronto Scarborough, Toronto, Ontario, Canada. dpare@psychexperiments.com
Behavior Research Methods
|July 10, 2009
Summary
Web-based research reliably collects normative data, but familiarity ratings differ online versus in a lab. This study explored research location effects on image rating tasks, finding online familiarity ratings higher in familiar online settings.
Area of Science:
- Cognitive Psychology
- Psychological Methods
Background:
- Web-based research offers potential for large-scale data collection.
- Previous studies indicate web-based research can replicate lab findings.
- Gaps remain in understanding limitations and differences in online data collection.
Purpose of the Study:
- To investigate the impact of research location (laboratory vs. online) on normative data collection.
- To compare object familiarity and visual complexity ratings across different environments.
- To identify potential discrepancies in data based on research setting.
Main Methods:
- Randomly assigned participants to either a laboratory or online condition.
- Collected ratings for 593 photorealistic images on object familiarity and visual complexity.
- Analyzed response latencies and inter-rater agreement across conditions.
Main Results:
- Normative data collected online demonstrated reliability.
- Online familiarity ratings were significantly higher than lab ratings.
- This location-based difference was specific to familiarity tasks, not visual complexity.
Conclusions:
- Online data collection is a reliable method for normative data.
- Research location significantly influences subjective ratings like familiarity.
- Familiarity effects in online research warrant consideration for experimental design.
