Web-based image norming: how do object familiarity and visual complexity ratings compare when collected in-lab versus online?

It has been demonstrated previously that, for some experimental paradigms, Web-based research can reliably replicate lab-based results. Yet questions remain as to what types of research can be reproduced, and where differences arise when they cannot be. The present article examines the effect of research location (laboratory vs. online) on normative data collection tasks. Specifically, participants were randomly assigned to a laboratory or online condition and were asked to rate 593 photorealistic images on the basis of object familiarity (N=103) and object visual complexity (N=98). Dependent measures were compared across location conditions, including response latencies and image rating agreement. Our results suggest that norming data collected online are reliable, but an interesting interplay between task type and research location was observed. Specifically, we found that participating online (i.e., a more familiar environment) leads to systematically higher familiarity ratings than in the lab (i.e., an unfamiliar environment). These differences are not found when the alternate complexity rating task is used.