Iterative alignment discovery of speech-associated neural activity.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computer Science

Background:

• Brain-computer interfaces (BCIs) offer speech restoration for neurological disorders.
• Accurate temporal alignment of neural activity and speech output is crucial for BCI training.
• A key challenge is training BCIs for patients unable to produce audible speech due to lack of ground truth.

Purpose of the Study:

• To introduce an iterative algorithm, iterative alignment discovery dynamic time warping (IAD-DTW), for neural voice activity detection (nVAD).
• To enable the discovery of precise alignments between electrocorticographic (ECoG) neural responses and speech attempts.
• To facilitate the training of speech BCIs for individuals with impaired speech production.

Main Methods:

• Developed IAD-DTW, integrating dynamic time warping (DTW) into a deep neural network (DNN) loss function.
• Tested IAD-DTW on able-bodied individuals with ECoG recordings during speech tasks.
• Simulated lack of ground truth by perturbing temporal correspondence between neural data and speech signals.

Main Results:

• IAD-DTW accurately predicted speech onset and duration.
• The algorithm demonstrated robustness, with <1% accuracy degradation even under maximal temporal misalignments.
• Performance remained high despite simulated absence of ground truth data.

Conclusions:

• IAD-DTW is computationally efficient and easily integrated into existing DNN-based nVAD systems.
• This method allows for the training of speech BCIs using ECoG data from non-vocal patients.
• Enables BCI development for conditions like Locked-In Syndrome, significantly advancing speech restoration possibilities.