Mapping multidimensional content representations to neural and behavioral expressions of episodic memory

Area of Science:

• Neuroscience
• Cognitive Science
• Neuroimaging

Background:

• Episodic memories are represented by distributed neural activity patterns.
• Previous neuroimaging studies decoded simple stimulus properties, not rich memory content.
• Semantic encoding models can characterize multidimensional information in episodic memories.

Purpose of the Study:

• To build and apply semantic encoding models to reconstruct episodic memory content.
• To investigate the neural representation of multidimensional memory information.
• To compare reconstructions during visual perception versus memory recall.

Main Methods:

• Functional magnetic resonance imaging (fMRI) in four human subjects.
• Development of semantic encoding models from fMRI data.
• Reconstruction of natural scene image content from brain activity during viewing and recall.
• Application of natural language processing to verbal recall data.

Main Results:

• Multidimensional semantic information was reconstructed from visual and lateral parietal cortices during viewing and recall.
• Visual cortex reconstructions were more accurate for viewed images; lateral parietal reconstructions were similar for viewed and recalled images.
• fMRI reconstructions matched subjects' verbal descriptions, with ventral temporal cortex reconstructions aligning better with own verbal recall.
• Encoding models transferred across subjects, enabling reconstruction of memories using independent subject data.

Conclusions:

• Multidimensional and idiosyncratic memory representations can be successfully reconstructed.
• Visual cortical and lateral parietal regions show differential sensitivity to external visual information versus internal memory generation.
• Semantic encoding models offer a powerful tool for decoding complex memory content.