Related Experiment Video
Updated: Aug 12, 2025
09:44
Methods to Test Visual Attention Online
Published on: February 19, 2015
12.0K
From Lab-Testing to Web-Testing in Cognitive Research: Who You Test is More Important than how You Test
Kim Uittenhove1,2, Stephanie Jeanneret2, Evie Vergauwe2
1University of Lausanne, Switzerland.
Journal of Cognition
|February 1, 2023
Summary
Choosing the right participant pool is more critical than the testing method for data quality in cognitive psychology research. Web-testing offers acceptable data quality, but participant pool selection significantly impacts results.
Area of Science:
- Cognitive Psychology
- Research Methodology
- Human-Computer Interaction
Background:
- Transitioning to web-testing in cognitive psychology presents challenges in maintaining data quality comparable to traditional lab-testing.
- Distinguishing the impact of testing modality versus participant pool on data quality is crucial for online research validity.
Purpose of the Study:
- To compare data quality between web-testing and lab-testing modalities.
- To evaluate the influence of different participant pools (MTurk, Prolific, university students) on data quality.
- To provide guidance on selecting participant pools for online cognitive psychology studies.
Main Methods:
- A working memory task was administered to 196 MTurk, 300 Prolific, and 255 University of Geneva participants.
- Within the university student group, 215 were web-tested and 40 were lab-tested.
- Data quality was assessed using reaction time, accuracy, anomalous values, and two behavioral benchmark effects.
Main Results:
- Participant pool significantly impacts data quality more than the testing modality.
- Web-testing resulted in a minor, acceptable decrease in data quality compared to lab-testing.
- Prolific participants yielded data quality comparable to web-tested students; MTurk participants showed significantly different results.
Conclusions:
- Web-testing is a viable method for cognitive psychology research, even with complex tasks.
- Careful selection of online participant pools is essential for ensuring research validity.
- The choice of participant pool (e.g., Prolific vs. MTurk) has a greater effect on data quality than the testing environment.
Related Concept Videos
01:20Introduction to Cognitive Psychology
572
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
572
01:14Decision Making: Traditional Method
4.1K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.1K
01:17Cognitivism
1.5K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.5K
01:13Psychology as a Science
813
Psychology, as a scientific discipline, aims to understand the mind and behavior through rigorous and systematic methods. The foundation of psychological research is evidence-based, relying heavily on the scientific method to derive and validate knowledge. This structured approach ensures that findings are reliable, valid, and applicable to broader contexts.
The scientific method in psychology involves six critical steps: making observations, formulating hypotheses, conducting tests, analyzing...
The scientific method in psychology involves six critical steps: making observations, formulating hypotheses, conducting tests, analyzing...
813
01:22Case Studies
11.7K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
11.7K
01:21Cognitive Learning
472
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
472