Subject-Agnostic Transformer-Based Neural Speech Decoding from Surface and Depth Electrode Signals.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Biomedical Engineering

Background:

• Speech decoding from neural signals is crucial for neuroprosthetics.
• Prior methods often limited to 2D electrode grids (ECoG) and single-patient data.
• Need for models accommodating diverse electrode types (ECoG, sEEG) and multiple participants.

Purpose of the Study:

• Develop a deep learning model for speech decoding using intracranial electrodes.
• Design an architecture compatible with both surface (ECoG) and depth (sEEG) electrodes.
• Enable training on multi-participant data with variable electrode placements for generalizability.

Main Methods:

• Proposed a novel transformer-based model architecture, SwinTW.
• Leveraged 3D electrode locations for arbitrary positioning, moving beyond 2D grid constraints.
• Trained both subject-specific and multi-patient models.

Main Results:

• Subject-specific SwinTW models achieved high decoding accuracy (PCC=0.817 with ECoG, PCC=0.798 with sEEG).
• Incorporating diverse electrode types improved performance (PCC=0.838).
• Multi-patient models demonstrated strong generalizability on unseen participants (PCC=0.765).

Conclusions:

• SwinTW enables flexible use of clinically optimal electrode placements for speech neuroprostheses.
• Generalizable multi-patient models can be applied to new patients without paired acoustic-neural data.
• Advances neuroprosthetic capabilities for individuals with speech disabilities.