从fMRI中检索和重建概念上相似的图像,使用隐性扩散模型和神经启发的大脑解码模型
Matteo Ferrante1, Tommaso Boccato1, Luca Passamonti2
1Department of Biomedicine and Prevention, University of Rome, Tor Vergata, Rome, Italy.
Journal of neural engineering
|June 17, 2024
概括
这项研究引入了一种新的大脑解码方法,使用语义相似性将大脑活动映射到视觉特征上. 这种方法成功地检索并生成与感知内容相匹配的图像,推进认知神经科学和人工智能.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 认知神经科学 认知神经科学
背景情况:
- 大脑解码旨在从大脑活动中推断出心理状态.
- 现有的方法往往缺乏对刺激的语义和上下文理解的关注.
研究的目的:
- 提出一种新的大脑解码方法,利用语义和上下文相似性.
- 开发一个深度学习管道,将大脑活动映射到视觉刺激的语义特征.
主要方法:
- 使用功能磁共振成像 (fMRI) 数据集与自然图像刺激.
- 创建了一个线性脑到特征模型,将fMRI活动映射到预训练的神经网络中的语义特征.
- 采用近邻策略进行特征分类和图像检索/生成,使用隐性扩散模型.
主要成果:
- 在三个fMRI数据集 (通用对象解码,BOLD5000,NSD) 中成功展示了语义分类和图像检索.
- 在图像相似性评估中,在人类评估中获得了超过80%的准确性.
- 展示了产生与原始视觉内容相匹配的新奇图像的能力.
结论:
- 可测量的神经相关值可以线性地映射到神经网络的隐藏空间,用于图像合成.
- 语义和上下文相似性为大脑解码提供了一个强大的新方向.
- 这些发现对理解视觉感知和推进人工智能有重大影响.
相关概念视频
Magnetic Resonance Imaging
5.1K
Magnetic resonance imaging (MRI) is a noninvasive medical imaging technique based on a phenomenon of nuclear physics discovered in the 1930s, in which matter exposed to magnetic fields and radio waves was found to emit radio signals. In 1970, a physician and researcher named Raymond Damadian noticed that malignant (cancerous) tissue gave off different signals than normal body tissue. He applied for a patent for the first MRI scanning device in clinical use by the early 1980s. The early MRI...
5.1K
Brain Imaging
225
Brain imaging technologies provide critical insights into both the structure and function of the human brain, enabling medical professionals and researchers to diagnose, study, and treat neurological disorders or psychiatric disorders more effectively.
These technologies include computerized axial tomography (CAT or CT scans), positron-emission tomography (PET scans), magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and Transcranial Magnetic...
These technologies include computerized axial tomography (CAT or CT scans), positron-emission tomography (PET scans), magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and Transcranial Magnetic...
225


