Towards direct speech synthesis from ECoG: A pilot study.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Speech Technology

Background:

• Current Brain-Computer Interfaces (BCIs) often rely on cumbersome spelling paradigms.
• Direct speech synthesis from neural activity offers a more natural communication method.
• Electrocorticography (ECoG) provides high-resolution intracranial neural data.

Purpose of the Study:

• To demonstrate real-time speech resynthesis from ECoG signals.
• To explore the feasibility of reconstructing audio from temporal lobe neural activity.
• To establish a foundation for speech synthesis from imagined speech.

Main Methods:

• Utilized electrocorticography (ECoG) intracranial activity from temporal areas in a single participant.
• Reconstructed audio magnitude spectrograms from recorded neural data.
• Synthesized audio waveforms from the reconstructed spectrograms.

Main Results:

• Achieved significant correlations between original and reconstructed audio spectrograms.
• Demonstrated significant correlations between original and reconstructed temporal waveforms.
• Successfully resynthesized speech in real-time from neural activity.

Conclusions:

• ECoG signals from temporal areas can be used for real-time speech resynthesis.
• This pilot study is a crucial first step towards synthesizing speech from neural activity, including imagined speech.
• The findings support the potential for more natural BCIs for communication.