Reading visually embodied meaning from the brain: Visually grounded computational models decode visual-object mental...

Area of Science:

• Cognitive Neuroscience
• Computational Linguistics
• Neuroimaging

Background:

• Embodiment theory posits mental imagery of objects engages perceptual brain circuits.
• The extent of visual imagery in daily thought and its neural encoding remain unclear.

Purpose of the Study:

• To investigate if fMRI patterns from reading object names reflect embodied visual-object representations.
• To decode these representations using computational image-based semantic models.

Main Methods:

• Applied image-based and text-based semantic models to fMRI data.
• Utilized representational similarity analysis to map brain activity to models.
• Employed an unsupervised decoding algorithm for classifying neural representations.

Main Results:

• Ventral-temporal and lateral-occipital regions showed strong correlation with image models.
• Posterior-parietal, lateral-temporal, and inferior-frontal regions correlated better with text models.
• Achieved high accuracy (8/10) in decoding embodied visual representations.

Conclusions:

• Neural representations of object words capture latent visual-semantic structure.
• Combining multimodal data enhances modeling of human-like semantic representations.
• Findings support the embodied nature of semantic memory and offer insights into neural encoding.