Protocol to perform offline ECoG brain-to-text decoding for natural tonal sentences

Area of Science:

• Neuroscience
• Computational Linguistics
• Biomedical Engineering

Background:

• Decoding speech from neural signals is crucial for brain-computer interfaces.
• Existing methods often lack efficiency or language specificity.
• Mandarin speech presents unique challenges due to its tonal nature.

Purpose of the Study:

• To present a comprehensive protocol for decoding Mandarin sentences from invasive neural recordings.
• To establish a robust brain-to-text framework for Mandarin speech.
• To detail the necessary steps for both material preparation and decoding procedures.

Main Methods:

• Designing a Mandarin sentence corpus and preparing electrocorticography (ECoG) recording systems.
• Implementing data preprocessing, electrode selection, and speech detection algorithms.
• Utilizing syllable and tone decoding combined with language modeling for accurate translation.

Main Results:

• The protocol outlines a systematic approach to neural decoding of Mandarin speech.
• It covers all stages from data acquisition to performance evaluation.
• Specific metrics for assessing decoding performance are detailed.

Conclusions:

• This protocol provides a reproducible method for brain-to-text communication in Mandarin.
• It lays the groundwork for developing advanced assistive technologies for individuals with speech impairments.
• Further research can build upon this framework to enhance decoding accuracy and expand language capabilities.