Related Experiment Video
Updated: Jan 2, 2026
08:15
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
1.1K
Motion capture-based animated characters for the study of speech-gesture integration.
Jens Nirme1, Magnus Haake2, Agneta Gulz2
1Lund University Cognitive Science, Lund, Sweden. jens.nirme@lucs.lu.se.
Behavior Research Methods
|December 12, 2019
Summary
Digitally animated characters, created using motion capture, enable precise manipulation of gestures for speech-gesture integration research. This method facilitates experiments on detecting speech-gesture synchrony.
Area of Science:
- Cognitive Science
- Human-Computer Interaction
- Linguistics
Background:
- Understanding how humans integrate auditory speech with visual cues like gestures is crucial for communication research.
- Previous research faced limitations in isolating and manipulating specific gesture features for experimental control.
Purpose of the Study:
- To present a novel workflow combining motion capture and 3D animation for detailed gesture manipulation.
- To enable the creation of controlled stimuli for investigating speech-gesture integration and synchrony detection.
Main Methods:
- Utilizing motion capture technology to record naturalistic gestures.
- Employing 3D animation to create characters and precisely manipulate captured gesture data (e.g., temporal shifting).
- Developing experimental paradigms to present these manipulated gestures alongside speech stimuli.
Main Results:
- Demonstrated the feasibility of isolating and modifying individual gesture strokes.
- Successfully created stimuli for an experiment on implicit speech-gesture synchrony detection.
- Showcased the workflow's applicability for generating ecologically valid gesture sequences.
Conclusions:
- The presented motion capture and 3D animation approach offers a powerful tool for experimental research on multimodal communication.
- This methodology allows for fine-grained control over visual gesture features, advancing the study of speech-gesture integration.
- The workflow is broadly applicable to various research questions within human-computer interaction and cognitive science involving gestural communication.
Related Concept Videos
01:14Muscles for Facial Expressions
4.5K
The craniofacial muscles are a collection of approximately 20 thin skeletal muscles situated beneath the skin of the face and scalp. These muscles, primarily responsible for the vast array of human facial expressions, originate from the bones or fibrous structures of the skull and extend outwards to connect with the skin. While most skeletal muscles in the body are enveloped in thick fascia, facial muscles generally have a more delicate fascial covering, with the buccinator muscle being a...
4.5K
01:24Muscle Coordination and Action
2.9K
Muscle coordination is a complex and finely tuned process essential for smooth and purposeful movements like flexion, extension, adduction, abduction, and rotation. The human body orchestrates the actions of various muscles working in concert, each with a specific role. Four functional types describe how muscles work together: agonist, antagonist, synergist, and fixator.
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
2.9K
00:51Anatomical Movements
14.6K
Anatomical movements refer to the various actions or motions that can be performed by the body's joints and muscles. These movements are described using specific terms to provide a standardized way of discussing and understanding the range of motion at different joints.
Here are some common anatomical movements:
Flexion and extension motions are in the sagittal (anterior–posterior) plane of motion. These movements take place at the shoulder, hip, elbow, knee, wrist,...
Here are some common anatomical movements:
Flexion and extension motions are in the sagittal (anterior–posterior) plane of motion. These movements take place at the shoulder, hip, elbow, knee, wrist,...
14.6K