Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced...

Area of Science:

• Cognitive Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Human behavior involves continuous stimulus-response (S-R) transformations, but understanding these processes in complex, real-world tasks is challenging.
• Traditional methods often discretize time, limiting insights into continuous neural dynamics.
• Deep neural networks (DNNs) offer a powerful framework for modeling neural processes, but their alignment with human cognition requires further investigation.

Purpose of the Study:

• To investigate whether advanced DNNs better capture neural representations of time-continuous S-R transformations.
• To compare the predictive accuracy of DNN features at different developmental stages within an encoding model.
• To assess the hierarchical alignment between DNN internal representations and brain functional organization.

Main Methods:

• Collected functional magnetic resonance imaging (fMRI) data from 23 participants playing arcade video games.
• Utilized DNN-based encoding models to predict human brain activity from game stimuli.
• Compared prediction accuracy using features from three DNNs at varying levels of sophistication.

Main Results:

• The most advanced DNN demonstrated superior prediction accuracy for neural responses.
• This advanced DNN also showed closer hierarchical alignment with the brain's functional organization.
• Findings suggest a progression of S-R transformations along the dorsal visual stream and into motor regions.

Conclusions:

• Advanced DNNs provide a more accurate and hierarchically aligned representation of neural activity during complex visuomotor tasks.
• This approach enhances the characterization of continuous S-R transformations.
• Machine learning, particularly DNNs, holds significant potential for advancing cognitive neuroscience research using ecologically valid tasks.