Transfer learning via distributed brain recordings enables reliable speech decoding
Aditya Singh1,2, Tessy Thomas1,2, Jinlong Li1,2,3
1Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, USA.
Abstract:
Speech brain-computer interfaces (BCIs) combine neural recordings with large language models to achieve real-time intelligible speech. However, these decoders rely on dense, intact cortical coverage and are challenging to scale across individuals with heterogeneous brain organization. To derive scalable transfer learning strategies for neural speech decoding, we used minimally invasive stereo-electroencephalography recordings in a large cohort performing a demanding speech motor task. A sequence-to-sequence model enabled decoding of variable-length phonemic sequences prior to and during articulation. This enabled development of a cross-subject transfer learning framework to isolate shared latent manifolds while enabling individual model initialization. The group-derived decoder significantly outperformed models trained on individual data alone, enabling decoding robustness despite variable coverage and activation. These results highlight a pathway toward generalizable neural prostheses for speech and language disorders by leveraging large-scale intracranial datasets with distributed spatial sampling and shared task demands.
More Related Videos
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
05:48Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
Related Concept Videos
Improving Translational Accuracy
Improving Translational Accuracy
Higher Mental Functions of the Brain: Language
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
