Related Experiment Video
Updated: Jan 4, 2026
08:15
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
1.1K
The quantification of gesture-speech synchrony: A tutorial and validation of multimodal data acquisition using
Wim Pouw1,2, James P Trujillo3,4, James A Dixon5
1Center for the Ecological Study of Perception and Action, University of Connecticut, Storrs, CT, USA. wimpouw@uconn.edu.
Behavior Research Methods
|October 30, 2019
Summary
Objective measurement of gesture kinematics is crucial for understanding speech-gesture synchronization. This study introduces efficient, low-cost video-based methods for analyzing gesture timing, offering a viable alternative to expensive systems.
Area of Science:
- Multimodal communication research
- Human-computer interaction
- Cognitive science
Background:
- Growing evidence shows synchronization between hand gestures and speech across various timescales.
- Previous gesture studies often relied on subjective, manual measurement of kinematic peaks.
- Objective, high-resolution kinematic analysis is essential for detailed study of gesture-speech coupling.
Purpose of the Study:
- To provide a tutorial on efficient methods for quantifying temporal properties of gesture kinematics.
- To address challenges in objective kinematic measurement within multimodal language research.
- To compare the performance of video-based motion tracking against traditional systems.
Main Methods:
- Tutorial on quantifying temporal properties of gesture kinematics.
- Comparison of two video-based motion tracking techniques (deep learning, pixel change).
- Validation against a high-performance wired motion tracking system (Polhemus Liberty) using a dataset of 392 gesture events.
Main Results:
- Video-based motion tracking methods demonstrate strong performance in estimating temporal properties of gesture kinematics.
- Deep learning and pixel change methods provide accurate temporal estimation of kinematic peaks.
- Videography methods offer a cost-effective alternative to expensive motion tracking systems.
Conclusions:
- Objective, video-based methods are effective for analyzing gesture kinematics.
- These methods facilitate the study of fine-grained temporal relationships between gesture and speech.
- Encourages wider adoption of objective kinematic analysis in gesture research.
Related Concept Videos
01:24Absolute Motion Analysis- General Plane Motion
496
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
496
01:25Relative Motion Analysis using Rotating Axes
835
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
835