An EEG-EMG-Based Hybrid Brain-Computer Interface for Decoding Tones in Silent and Audible Speech

Area of Science:

• Neuroscience
• Biomedical Engineering
• Speech and Language Technology

Background:

• Tonal variations are crucial for semantic meaning in languages like Mandarin.
• Limited research exists on neural signatures of tones and their decoding.
• Brain-computer interfaces (BCIs) offer potential for communication support for individuals with language disabilities.

Purpose of the Study:

• To systematically investigate the neural signatures of the four Mandarin tones.
• To explore the feasibility of decoding tones in silent and audible speech using a multimodal BCI.
• To assess the effectiveness of EEG and EMG signals for tone classification and speech restoration.

Main Methods:

• Utilized a multimodal BCI combining electroencephalography (EEG) and electromyography (EMG).
• Performed time-frequency analysis on EEG data to identify tone-dependent neural variations.
• Analyzed EMG signals from facial muscles (buccinator, mentalis) for tone-related differences.
• Employed Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) classifiers for tone classification.

Main Results:

• Significant tone-dependent neural activation patterns were observed in the frontal lobe (EEG) and specific facial muscles (EMG).
• Neural differences were more pronounced in audible speech compared to silent speech conditions.
• EEG temporal features achieved up to 72.43% accuracy for four-tone classification and over 90% for binary classification.
• The combined EEG and EMG approach yielded the highest decoding accuracy of 81.33%.

Conclusions:

• This research validates the feasibility of using a speech brain-computer interface (BCI) for tonal language impairment.
• The findings demonstrate a potential strategy for speech restoration in tonal languages.
• The study highlights the importance of considering tonal variations in BCI development for diverse populations.