Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to...

Area of Science:

• Neuroscience
• Cognitive Science
• Computer Science (Deep Learning)

Background:

• Functional magnetic resonance imaging (fMRI) and deep learning have advanced visual encoding and decoding.
• Human visual processing involves bidirectional information flow between primary and high-level visual cortices (bottom-up and top-down).
• Existing methods may not fully leverage these bidirectional flows for accurate visual decoding.

Purpose of the Study:

• To propose a novel method for decoding visual categories from fMRI data.
• To leverage the bidirectional information flow in the human visual system.
• To improve the accuracy of category decoding compared to existing approaches.

Main Methods:

• Developed a bidirectional recurrent neural network (BRNN) model inspired by bottom-up and top-down visual processing.
• Treated voxels from visual areas (V1-V4, LO) as sequential nodes for the BRNN.
• Integrated BRNN outputs with a fully connected softmax layer for category decoding.

Main Results:

• The proposed BRNN-based method significantly improved the accuracy of three-level category decoding from fMRI data.
• Experimental results demonstrated the method's ability to efficiently utilize hierarchical and bidirectional information.
• Comparative analysis confirmed the presence of correlative category representations due to bidirectional information flow.

Conclusions:

• The BRNN method effectively models bidirectional information flow in the visual cortex for enhanced fMRI decoding.
• This approach captures hierarchical, distributed, and complementary representations, aligning with and extending previous findings.
• The study highlights the importance of considering both bottom-up and top-down processing for understanding visual cognition via neuroimaging.