Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A Probabilistic Approach to Functional Organization Based on Extraoperative Electrocortical Stimulation Mapping.

Neurology·2026
Same author

Frontal cortex organization supporting audiovisual processing during naturalistic viewing.

Nature communications·2026
Same author

CL-316243 facilitates stable atherosclerotic plaque phenotypes in association with suppression of perivascular adipose tissue ferroptosis via upregulating C/EBPβ.

Redox biology·2026
Same author

Activating π-Electron Conjugated Networks of Self-Assembled Multilayers for 21.1% Efficiency Organic Solar Cells.

Angewandte Chemie (International ed. in English)·2026
Same author

On-site exposure to clinical epilepsy practice for experimental scientists engaged in epilepsy research: A pilot study by the ILAE commission on neurobiology.

Epilepsia open·2026
Same author

A Phase-2 Open-Label Trial of Cannabidiol to Treat Core and Associated Symptoms of Autism in Children and Adolescents Without Intellectual Disability.

Journal of child and adolescent psychopharmacology·2026
Same journal

Developing a binary communication protocol between biological neural networks using virtual white matter.

Journal of neural engineering·2026
Same journal

Spatiotemporally distinctive astrocytic and neuronal responses to repetitive intracortical microstimulation.

Journal of neural engineering·2026
Same journal

A neural mass modelling framework for evaluating EEG source localisation of seizure activity.

Journal of neural engineering·2026
Same journal

Functional and effective connectivity methods from SEEG for characterizing epileptogenic networks in refractory epilepsy: a comprehensive review and future directions.

Journal of neural engineering·2026
Same journal

Online decoding of rat self-paced locomotion speed from EEG using recurrent neural networks.

Journal of neural engineering·2026
Same journal

The seizure embedding map: A spatio-temporal transformer for comparing patients by ictal intracranial EEG features at scale.

Journal of neural engineering·2026
See all related articles

Related Experiment Video

Updated: Jun 2, 2025

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
11:14

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants

Published on: October 4, 2015

10.9K

Transformer-based neural speech decoding from surface and depth electrode signals.

Junbo Chen1, Xupeng Chen1, Ran Wang1

  • 1Electrical and Computer Engineering Department, New York University, 370 Jay Street, Brooklyn, NY 11201, United States of America.

Journal of Neural Engineering
|January 17, 2025
PubMed
Summary
This summary is machine-generated.

A new deep-learning model, SwinTW, decodes speech from neural signals using any electrode type and placement. This flexible model achieves high accuracy across multiple participants, even those unseen during training.

Keywords:
ECoGelectrocorticographicneural speech decodingneural speech prosthesisspeech synthesis

More Related Videos

Simultaneous Scalp Electroencephalography EEG, Electromyography EMG, and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding
11:25

Simultaneous Scalp Electroencephalography EEG, Electromyography EMG, and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding

Published on: July 26, 2013

43.2K
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.1K

Related Experiment Videos

Last Updated: Jun 2, 2025

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
11:14

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants

Published on: October 4, 2015

10.9K
Simultaneous Scalp Electroencephalography EEG, Electromyography EMG, and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding
11:25

Simultaneous Scalp Electroencephalography EEG, Electromyography EMG, and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding

Published on: July 26, 2013

43.2K
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.1K

Area of Science:

  • Neuroscience
  • Machine Learning
  • Biomedical Engineering

Background:

  • Speech decoding from neural signals is crucial for communication restoration in individuals with severe speech impairments.
  • Existing methods often rely on specific electrode configurations (e.g., 2D grids) and single-patient data, limiting generalizability.
  • There is a need for advanced deep-learning models capable of integrating diverse neural data from multiple participants with varied electrode placements.

Purpose of the Study:

  • To develop a novel deep-learning architecture, SwinTW, for speech decoding that accommodates both surface electrocorticography (ECoG) and stereotactic electroencephalography (sEEG) electrodes.
  • To enable training on multi-participant data with variable electrode configurations without subject-specific layers.
  • To achieve high speech decoding performance on unseen participants.

Main Methods:

  • Proposed a transformer-based model, SwinTW, leveraging the 3D cortical locations of arbitrarily positioned electrodes.
  • Trained subject-specific models using single-participant data and multi-subject models using data from multiple participants.
  • Evaluated performance using Pearson Correlation Coefficient (PCC) with ground truth spectrograms.

Main Results:

  • Subject-specific models with low-density ECoG data achieved a PCC of 0.817 across 43 participants, outperforming previous models.
  • Incorporating additional electrode types (strip, depth, grid) further improved performance to PCC = 0.838.
  • A single multi-subject model trained on 15 participants achieved comparable performance (PCC = 0.837) to individually trained models and generalized well to unseen participants (average PCC = 0.765).

Conclusions:

  • The SwinTW decoder effectively decodes speech from neural signals using diverse electrode types and placements, including depth electrodes.
  • The model's ability to train on multi-participant data and generalize to unseen subjects demonstrates its broad applicability and potential for clinical translation.
  • This approach paves the way for more personalized and effective brain-computer interfaces for communication restoration.