Reconstructing high-resolution visual perceptual images from human intracranial electrocorticography signals

Area of Science:

• Neuroscience
• Computer Vision
• Machine Learning

Background:

• Reconstructing visual perception from brain signals is a growing research area.
• Electrocorticography (ECoG) offers high-resolution intracranial signals for this purpose.
• No prior studies have reconstructed perceived images directly from human ECoG signals.

Purpose of the Study:

• To develop a novel pipeline for reconstructing perceived images from human ECoG signals.
• To evaluate the performance of the proposed method against state-of-the-art techniques.
• To explore the potential of unsupervised learning in decoding brain signals for visual reconstruction.

Main Methods:

• Developed a novel pipeline integrating Talairach coordinate alignment masked autoencoders (TA-MAE) with denoising diffusion probabilistic models.
• Utilized the spatiotemporal dynamics of human ECoG signals for high-resolution image restoration.
• Employed unsupervised learning for signal reconstruction, contrasting with label-guided methods.

Main Results:

• The proposed method significantly outperforms current state-of-the-art approaches in image appearance, structure, signal-noise ratio, and semantic consistency.
• Unsupervised learning-based reconstruction effectively captures low-dimensional brain signal representations compared to supervised methods.
• The findings suggest advantages of unsupervised decoding for exploring intrinsic mechanisms of vision.

Conclusions:

• The developed pipeline offers a generalizable framework for human ECoG-based visual reconstruction.
• Unsupervised decoding demonstrates superior performance and potential for brain signal analysis.
• This work pioneers the reconstruction of perceived images from human ECoG signals.