Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Hands can play equal roles in bimanual single-object rotations.

Acta psychologica·2026
Same author

Eyes don't lie: Eye tracking reveals whether an eyewitness saw the crime.

Acta psychologica·2026
Same author

Goals for kinematic coding: Insights from the posture-based motion planning perspective.

Physics of life reviews·2025
Same author

Acting with the feet and hands: Does one effector system dominate the other?

Acta psychologica·2024
Same author

Gaze Zone Classification for Driving Studies Using YOLOv8 Image Classification.

Sensors (Basel, Switzerland)·2024
Same author

The use of machine learning to understand the role of visual attention in multi-attribute choice.

Acta psychologica·2024

Related Experiment Video

Updated: Dec 13, 2025

Driving Simulation in the Clinic: Testing Visual Exploratory Behavior in Daily Life Activities in Patients with Visual Field Defects
11:12

Driving Simulation in the Clinic: Testing Visual Exploratory Behavior in Daily Life Activities in Patients with Visual Field Defects

Published on: September 18, 2012

17.7K

Does agreement mean accuracy? Evaluating glance annotation in naturalistic driving data.

Reinier J Jansen1, Sander T van der Kint2, Frouke Hermens2

  • 1SWOV Institute for Road Safety Research, P.O. Box 93113, 2509, AC, The Hague, The Netherlands. reinier.jansen@swov.nl.

Behavior Research Methods
|July 31, 2020
PubMed
Summary

Human annotation of driver glance direction in naturalistic driving studies is often inaccurate, falling below 50% for most locations. High annotator agreement does not ensure accuracy, necessitating experimental verification.

Keywords:
AccuracyAnnotationGlance perceptionNaturalistic drivingReliability

More Related Videos

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
07:36

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

Published on: November 30, 2018

16.2K
A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers
12:39

A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers

Published on: January 18, 2020

8.0K

Related Experiment Videos

Last Updated: Dec 13, 2025

Driving Simulation in the Clinic: Testing Visual Exploratory Behavior in Daily Life Activities in Patients with Visual Field Defects
11:12

Driving Simulation in the Clinic: Testing Visual Exploratory Behavior in Daily Life Activities in Patients with Visual Field Defects

Published on: September 18, 2012

17.7K
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
07:36

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

Published on: November 30, 2018

16.2K
A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers
12:39

A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers

Published on: January 18, 2020

8.0K

Area of Science:

  • Human-Computer Interaction
  • Transportation Safety
  • Computer Vision

Background:

  • Naturalistic driving studies utilize cameras to monitor driver behavior.
  • Human annotation of video data is the standard for training and evaluating computer vision algorithms.
  • The accuracy of human annotation for driver behavior analysis remains uncertain.

Purpose of the Study:

  • To evaluate the accuracy of human annotation for driver glance direction.
  • To compare instructed and actual glance directions of truck drivers with annotated data.
  • To investigate the impact of clustering on annotation accuracy and detail.

Main Methods:

  • Collected data on truck drivers' instructed and actual glance directions.
  • Compared driver glance directions with human-generated annotations.
  • Analyzed annotation accuracy across different locations.
  • Examined the effect of clustering locations on annotation accuracy and detail.

Main Results:

  • Annotation accuracy for driver glance direction is often below 50%.
  • Clustering locations can improve accuracy but reduces annotation detail.
  • High agreement between annotators does not correlate with high accuracy.

Conclusions:

  • The accuracy of human annotation in naturalistic driving studies requires more frequent experimental verification.
  • Annotation strategies, including clustering, should be chosen based on the specific research purpose.
  • Relying solely on annotator agreement can be misleading regarding data accuracy.