Related Experiment Video
Updated: Aug 6, 2026
Methods to Test Visual Attention Online
Published on: February 19, 2015
The timing mega-study: comparing a range of experiment generators, both lab-based and online
David Bridges1, Alain Pitiot2, Michael R MacAskill3,4
1School of Psychology, University of Nottingham, Nottingham, UK.
This study evaluated the timing precision of behavioral research software. Lab-based experiments showed millisecond precision with packages like PsychoPy, while online studies offered slightly more variability but remained viable for many research applications.
Area of Science:
- Behavioral Sciences
- Cognitive Psychology
- Human-Computer Interaction
Background:
- Behavioral sciences research relies on software for precise stimulus presentation and response time measurement.
- Existing timing performance data for popular research software packages is scarce.
- Accurate timing is crucial for sub-millisecond precision in behavioral experiments.
Purpose of the Study:
- To comprehensively evaluate the timing precision and accuracy of various behavioral research software packages.
- To compare performance across different software, operating systems, and online/lab-based environments.
- To provide empirical data to guide researchers in selecting appropriate software for their needs.
Main Methods:
- A wide-ranging study measured visual and auditory stimulus timing and response times using a Black Box Toolkit.
- Ten popular software packages (PsychoPy, E-Prime®, NBS Presentation®, Psychophysics Toolbox, OpenSesame, Expyriment, Gorilla, jsPsych, Lab.js, Testable) were tested.
- Tests were conducted across Windows, macOS, Ubuntu, and various web browsers for online studies, accumulating over 110,000 trials.
Main Results:
- Lab-based experiments: Psychtoolbox, PsychoPy, Presentation, and E-Prime demonstrated the highest timing precision (mean < 1 ms).
- Operating systems: Ubuntu generally offered slightly better precision than Windows; macOS performed worst for visual stimuli.
- Online studies: PsychoPy and Gorilla showed the best performance, achieving near-millisecond precision in some configurations; response time precision was generally under 10 ms across most packages.
Conclusions:
- Dedicated lab-based software like PsychoPy and E-Prime offer high timing precision crucial for behavioral research.
- Online research methods, while exhibiting more variability, are suitable for many studies with careful consideration of error sources.
- Researchers must validate timing performance with their specific stimuli and computer configurations due to observed variability.
More Related Videos
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
07:25Simultaneous Monitoring of Wireless Electrophysiology and Memory Behavioral Test as a Tool to Study Hippocampal Neurogenesis
Published on: August 20, 2020
