Decoding Speech With Integrated Hybrid Signals Recorded From the Human Ventral Motor Cortex

Area of Science:

• Neuroscience
• Biomedical Engineering
• Speech Communication

Background:

• Brain computer interfaces (BCIs) are crucial for restoring communication in locked-in patients.
• Single/multi-unit activity (SUA/MUA), local field potential (LFP), and electrocorticography (ECoG) are potential neural signals for BCIs.
• The optimal neural signal or combination for decoding speech production is not yet established.

Purpose of the Study:

• To determine the most effective neural signal modality for decoding speech production.
• To investigate if combining SUA, LFP, and ECoG signals enhances speech decoding accuracy.
• To assess the feasibility of simultaneous multi-scale neuronal activity recording from a localized cortical area.

Main Methods:

• Fabricated a novel 7x13 mm electrode array with microneedle and macro contacts for simultaneous SUA, LFP, and ECoG recording.
• Constructed feature vectors using spike frequency (SUA) and event-related spectral perturbation (ECoG/LFP).
• Inputted feature vectors into a decoder to assess speech phoneme decoding accuracy.

Main Results:

• Decoding accuracy for five spoken vowels reached 59% when averaged across subjects.
• The highest decoding accuracy was achieved by combining multiple signal features, optimized per subject.
• Multi-scale signals were found to convey complementary information crucial for speech articulation.

Conclusions:

• Simultaneous recording of multi-scale neuronal activities enhances speech decoding accuracy.
• Combining neural signals from a limited cortical area offers advantages for speech-assisting BCI implementation.
• This approach holds promise for advancing communication restoration technologies for individuals with severe motor impairments.