基于变压器的神经语音解码从表面和深度电极信号
Junbo Chen1, Xupeng Chen1, Ran Wang1
1Electrical and Computer Engineering Department, New York University, 370 Jay Street, Brooklyn, NY 11201, United States of America.
Journal of neural engineering
|January 17, 2025
概括
一个新的深度学习模型,SwinTW,使用任何电极类型和位置从神经信号解码语音. 这种灵活的模型在多个参与者之间实现了高精度,即使是在培训期间看不见的参与者也是如此.
科学领域:
- 神经科学是一个神经科学.
- 机器学习 机器学习
- 生物医学工程 生物医学工程
背景情况:
- 来自神经信号的语音解码对于严重语音障碍患者的沟通恢复至关重要.
- 现有的方法通常依赖于特定的电极配置 (例如,2D网格) 和单个患者的数据,限制了概括性.
- 需要先进的深度学习模型,能够整合来自多个参与者的多种神经数据,并使用不同的电极放置.
研究的目的:
- 开发一种新的深度学习架构,SwinTW,用于语音解码,可以容纳表面电皮图 (ECoG) 和立体电脑图 (sEEG) 电极.
- 为了使多参与者数据的训练具有可变的电极配置,没有特定主题的层.
- 为了在看不见的参与者身上实现高语音解码性能.
主要方法:
- 提出了一个基于变压器的模型,SwinTW,利用任意放置的电极的3D皮质位置.
- 训练有素的主体特定模型使用单参与者数据和多主体模型使用多个参与者的数据.
- 使用皮尔森相关系数 (PCC) 评估的性能与地面真相光谱图.
主要成果:
- 具有低密度ECoG数据的特定学科模型在43名参与者中实现了0.817的PCC,超过了以前的模型.
- 结合额外的电极类型 (条形,深度,网格) 进一步提高了PCC = 0.838.8的性能.
- 一个在15名参与者身上训练的单个多主体模型实现了与个别训练的模型相比的性能 (PCC = 0.837) 并对未见的参与者进行了很好的概括 (平均PCC = 0.765).
结论:
- 该SwinTW解码器有效地解码语音从神经信号使用不同的电极类型和放置,包括深度电极.
- 该模型能够在多参与者数据上进行训练,并对未见过的主题进行概括,这表明其广泛适用性和临床转化潜力.
- 这种方法为更个性化和更有效的脑计算机接口铺平了道路,以恢复通信.
更多相关视频
11:14A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
Published on: October 4, 2015
10.9K
11:39Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
2.1K
相关概念视频
Hearing
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
Auditory Pathway
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
Neuronal Communication
Neurons, the fundamental units of the brain and nervous system, communicate through complex electrochemical signals that underpin all cognitive and bodily functions. This communication is primarily facilitated by a process involving the generation and propagation of an action potential along the axon of the neuron. When the internal electrical charge of a neuron surpasses a certain threshold, an action potential is triggered. This rapid change in voltage travels swiftly along the axon to the...
