Related Experiment Video
Updated: Jan 15, 2026
Assessment of Social Cognition in Non-human Primates Using a Network of Computerized Automated Learning Device ALDM Test Systems
Published on: May 5, 2015
Assessing attentiveness and cognitive engagement across tasks using video-based action understanding in non-human
Sin-Man Cheung1, Adam Neumann1, Thilo Womelsdorf2
1Dep. of Psychology, Vanderbilt University, Nashville, TN 37240, United States.
Background:
Distractibility and attentiveness are cognitive states that are expressed through observable behavior, but how behavioral features can be used to quantify these cognitive states has remained poorly understood. Video-based analysis promises to be a versatile tool to quantify the behavioral features that reflect subject-specific distractibility and attentiveness and are diagnostic of cognitive states.
New Method:
We describe an analysis pipeline that classifies cognitive states using a 2-camera set-up for video-based estimation of attentiveness and screen engagement in nonhuman primates performing cognitive tasks. The procedure reconstructs 3D poses from 2D labeled DeepLabCut videos, reconstructs the head/yaw orientation relative to a task screen, and arm/hand/wrist engagements with task objects, to segment behavior into an attentiveness and engagement score.
Results:
Performance of different cognitive tasks was robustly classified from video within a few frames, reaching > 90 % decoding accuracy with ≤ 3 min long time segments. The analysis procedure allows adjusting thresholds for segmenting subject-specific movements for a time-resolved scoring of attentiveness and screen engagement.
Comparison With Existing Methods:
Current methods also extract poses and segment action units; however, they haven't been combined into a framework that enables subject-adjusted thresholding for specific task contexts. This integration is needed for inferring cognitive state variables and differentiating performance across various tasks.
Conclusion:
The proposed method integrates video segmentation, scoring of attentiveness and screen engagement, and classification of task performance at high temporal resolution. This integrated framework provides a tool for assessing attention functions from video.
