Word classification across speech modes from low-density electrocorticography signals.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Speech Science

Background:

• Speech brain-computer interfaces (BCIs) offer communication alternatives for non-speaking individuals.
• Decoding attempted speech is advanced, but imagined speech decoding remains a challenge.
• Neural mechanisms linking different speech modes are not well understood.

Purpose of the Study:

• To investigate the decoding of imagined speech using electrocorticography (ECoG).
• To explore the relationship between different speech modes (speaking, listening, imagining, mouthing, reading).
• To assess the effectiveness of transferring and augmenting decoding models across speech modes.

Main Methods:

• Collected low-density ECoG signals from ten participants during a word repetition task.
• Developed linear discriminant analysis models to classify five words across different speech modes.
• Investigated cross-modal model transfer and augmentation to improve decoding accuracy.

Main Results:

• Performed speech yielded the highest classification accuracy, followed by listening, mouthing, imagining, and reading.
• Model transfer and augmentation significantly improved decoding performance across speech modes.
• Transferring models from performed or perceived speech boosted imagined speech decoding in most participants, enabling above-chance decoding in some.

Conclusions:

• Cross-modal model transfer is a promising strategy to enhance imagined speech decoding for BCIs.
• Leveraging patterns from performed and perceived speech can significantly improve the performance of imagined speech BCIs.
• This approach has the potential to accelerate the development of more effective speech BCIs for users who prefer imagined speech.