Unraveling the Differential Efficiency of Dorsal and Ventral Pathways in Visual Semantic Decoding

Visual semantic decoding aims to extract perceived semantic information from the visual responses of the human brain and convert it into interpretable semantic labels. Although significant progress has been made in semantic decoding across individual visual cortices, studies on the semantic decoding of the ventral and dorsal cortical visual pathways remain limited. This study proposed a graph neural network (GNN)-based semantic decoding model on a natural scene dataset (NSD) to investigate the decoding differences between the dorsal and ventral pathways in process various parts of speech, including verbs, nouns, and adjectives. Our results indicate that the decoding accuracies for verbs and nouns with motion attributes were significantly higher for the dorsal pathway as compared to those for the ventral pathway. Comparative analyses reveal that the dorsal pathway significantly outperformed the ventral pathway in terms of decoding performance for verbs and nouns with motion attributes, with evidence showing that this superiority largely stemmed from higher-level visual cortices rather than lower-level ones. Furthermore, these two pathways appear to converge in their heightened sensitivity toward semantic content related to actions. These findings reveal unique visual neural mechanisms through which the dorsal and ventral cortical pathways segregate and converge when processing stimuli with different semantic categories.