Real-time decoding of full-spectrum Chinese using brain-computer interface

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computational Linguistics

Background:

• Speech brain-computer interfaces (BCIs) are crucial for restoring communication in patients with anarthria due to conditions like ALS or stroke.
• Existing speech decoding research primarily focuses on English, facing challenges with tonal, monosyllabic languages like Mandarin Chinese.

Purpose of the Study:

• To develop and evaluate a real-time Mandarin Chinese speech BCI capable of decoding monosyllabic units directly from neural signals.
• To address the limitations of current phoneme-driven approaches for tonal languages.

Main Methods:

• Utilized a 256-channel microelectrocorticographic (ECoG) BCI system.
• Developed a direct syllable neural decoding architecture for Mandarin Chinese.
• Tested the system in a single-character reading task.

Main Results:

• Achieved robust decoding of 394 distinct Mandarin syllables directly from neural signals.
• Demonstrated a median syllable identification accuracy of 71.2% in a single-character reading task.
• Successfully implemented real-time sentence decoding leveraging the syllable decoder.

Conclusions:

• The developed tonally integrated, direct syllable neural decoding approach is effective for Mandarin Chinese.
• This BCI system shows promise for restoring communication in tonal monosyllabic language speakers.
• Paves the way for comprehensive communication systems in languages with similar structures.