Decoding individual differences in mental information from human brain responses predicted by deep neural networks

Brain decoding using functional magnetic resonance imaging (fMRI) has high potential for evaluating unobservable mental information that varies across individuals in everyday situations. Although estimating individual differences in mental information using brain decoding could be effective for social applications, especially in neuromarketing, the physical constraints and high costs of fMRI limit the practical application in real-world settings. To overcome these limitations, we propose a framework to estimate individual differences in decodable mental information using brain responses predicted by deep neural networks (DNNs). In this framework, DNN-based models are trained to predict fMRI responses to natural movie scenes. Then, additional fMRI measurements are no longer required because mental information for new movie scenes can be decoded from the predicted responses instead of the measured ones. A large-scale analysis revealed that the proposed predicted-response decoding captures individual difference patterns consistent with those derived from conventional measured-response decoding across various cognitive domains. Of the 88 cognitive labels associated with movie scenes, individual difference patterns could be captured for more than 70% of the labels. This framework offers a promising approach for evaluating personal mental information with substantially reduced reliance on fMRI, thereby expanding the potential for social applications of brain decoding.