Road Hazard Stimuli: Annotated naturalistic road videos for studying hazard detection and scene perception

Driving requires vision, yet there is little empirical data about how vision and cognition support safe driving. It is difficult to study perception during natural driving because the experimental rigor required would be dangerous and unethical to implement on the road. The driving environment is complex, dynamic, and immensely variable, making it extremely challenging to accurately replicate in simulation. Our proposed solution is to study vision using stimuli which reflect this inherent complexity by using footage of real driving situations. To this end, we curated a set of 750 crowd-sourced video clips (434 hazard and 316 no-hazard clips), which have been spatially, temporally, and categorically annotated. These annotations describe where the hazard appears, what it is, and when it occurs. In addition, perceived dangerousness changes from moment to moment and is not a simple binary detection judgement. To capture this more granular aspect of our stimuli, we asked 48 observers to rate the perceived hazardousness of 1356 brief video clips taken from these 750 source clips on a continuous scale. These ratings span the entire scale, have high interrater agreement, and are robust to driving history. This novel stimulus set is not only useful for understanding drivers' ability to detect hazards, but is also a tool for studying dynamic scene perception and other aspects of visual function. While this stimulus set was originally designed for behavioral studies, researchers interested in other areas such as traffic safety or computer vision may also find this dataset a useful resource.