Related Experiment Video
Updated: Mar 23, 2026
11:15
fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals
Published on: May 23, 2017
7.7K
High-Resolution, Non-Invasive Imaging of Upper Vocal Tract Articulators Compatible with Human Brain Recordings
Kristofer E Bouchard1,2, David F Conant2,3, Gopala K Anumanchipalli2,3
1Biological Systems and Engineering Division & Computational Research Division, Lawrence Berkeley National Laboratories (LBNL), Berkeley, California, United States of America.
Plos One
|March 29, 2016
Summary
Researchers developed a noninvasive imaging system to track speech articulator movements and decode lip movements from brain activity, advancing speech motor control understanding and prosthetic development.
Area of Science:
- Neuroscience
- Speech Science
- Biomedical Engineering
Background:
- Understanding speech motor control requires correlating neural activity with articulator kinematics.
- Tracking internal vocal tract articulators (lips, jaw, tongue, larynx) simultaneously is challenging, especially during human electrophysiology.
Purpose of the Study:
- To develop and validate a noninvasive, multi-modal imaging system for monitoring vocal tract kinematics during speech.
- To analyze the relationship between articulator movements, acoustics, and neural activity.
- To decode speech kinematics from brain signals.
Main Methods:
- A novel noninvasive, multi-modal imaging system was developed to track vocal tract articulator kinematics.
- Data from six speakers producing nine American English vowels were analyzed using classification, regression, and non-negative matrix factorization.
- Electrocorticography (ECoG) data were used to predict lip kinematics.
Main Results:
- The system successfully monitored articulator kinematics non-invasively.
- Variability in the articulator-to-acoustic relationship across speakers was identified.
- Non-negative matrix factorization improved vowel classification accuracy.
- Lip kinematics were successfully predicted from ventral sensorimotor cortical activity.
Conclusions:
- The developed multi-modal system and analytic methods enable non-invasive monitoring and analysis of speech production.
- This study provides the first decoding of speech kinematics from electrocorticography.
- These advances are crucial for understanding the neural basis of speech and developing advanced vocal prosthetics.
