Neural substrates underlying multisensory stiffness perception via active touch and dynamic visual feedback

Humans perceive the physical properties of objects through active touch to acquire information that is unavailable by passive observation (e.g., pinching an object to estimate its stiffness). Previous functional neuroimaging studies have investigated neural representations of multisensory shape and texture perception using active touch and static images of objects. However, in active visuo-haptic perception, in addition to static visual information from the object itself, dynamic visual feedback of exploratory actions is a crucial cue. To integrate multisensory signals into a unitary percept, the brain must determine whether somatosensory sensation and dynamic visual feedback are congruent and caused by the same exploratory action. The influence of dynamic visual feedback during exploratory actions has not yet been examined, and the neural substrates for multisensory stiffness perception are still unknown. Here, we developed a functional magnetic resonance imaging-compatible device that enables users to perceive the stiffness of a virtual spring by pinching two finger plates and obtaining real-time visual feedback from the finger plate movements. After confirming the integration of visual and haptic cues in behavioral experiments, we investigated neural regions for multisensory stiffness signal processing and action-feedback congruency separately in two functional magnetic resonance imaging experiments. Modulating the stiffness level and contrasting bimodal/unimodal conditions revealed that multisensory stiffness information converged to the bilateral superior parietal lobules and supramarginal gyri, while congruent action-feedback conditions elicited significantly stronger neural responses in the left mid-cingulate cortex, left postcentral gyrus, and bilateral parietal opercula. Further analysis using dynamic causal modeling suggested top-down modulatory connections from the left mid-cingulate cortex to the bilateral parietal opercula and left postcentral gyrus when visual feedback was consistent with finger movements. Our results shed light on the neural mechanisms involved in estimating object properties from active touch and dynamic visual feedback, which may involve two distinct neural networks: one for multisensory signal processing and the other for action-feedback congruency judgment.