Neural decoding of speech using deep neural ensembles

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computer Science

Background:

• Speech brain-computer interfaces (BCIs) offer communication restoration for individuals with paralysis.
• Current decoding errors limit BCI performance, hindering widespread clinical adoption.
• Deep ensemble methods show promise in improving accuracy but lack real-time testing and understanding of performance under clinical constraints.

Purpose of the Study:

• To evaluate the real-time performance of deep ensemble methods in speech BCIs.
• To assess the impact of deep ensembles on word error rate in a large-vocabulary task.
• To investigate the resource-accuracy trade-offs of deep ensembles and explore computationally efficient alternatives.

Main Methods:

• Closed-loop testing of deep ensembles with intracortical microelectrode arrays in a participant.
• Analysis of deep ensemble performance across multiple participants, varying baseline error rates, training data size, and ensemble size.
• Development and evaluation of a computationally efficient pseudoensembling approach using test-time augmentation.

Main Results:

• Deep ensembles reduced word error rate from 33.7% to 26.0% in real-time speech decoding.
• Performance gains were analyzed concerning baseline error, dataset size, and ensemble size, highlighting resource-accuracy trade-offs.
• A novel pseudoensembling method achieved accuracy improvements with significantly reduced computational cost.

Conclusions:

• Deep ensemble methods can be effectively implemented in real-time speech BCIs, significantly improving decoding accuracy.
• The study provides crucial insights into the practical application and optimization of deep ensembles for clinical BCI deployment.
• Computationally efficient pseudoensembling offers a viable path to harness ensemble benefits with reduced resource requirements, advancing BCI technology.