Movement Sonification During Haptic Exploration Shifts Emotional Outcome Without Altering Texture Perception

Adding movement sonification to haptic exploration can change the perceptual outcome of a textured surface through multisensory processing. We hypothesized that auditory-evoked emotions influence the appraisal of textured surfaces, with corresponding changes reflected in cortical excitability.

Twelve participants actively rubbed two different textured surfaces (slippery and rough) either without movement sonification, or with pleasant or disagreeable movement sonification.

We found that sounds, whether agreeable or disagreeable, did not change the texture appraisal. However, the less pleasant surface was associated with a stronger negative hedonic valence, particularly when paired with disagreeable movement sonification. Time frequency analyses of electroencephalography (EEG) activities revealed a significant reduction in beta-band power [15-25 Hz] within the source-estimated sensorimotor and superior posterior parietal cortices when contrasting both pleasant and unpleasant sounds with the silent touch. This suggests that the primary somatosensory cortices together with the superior parietal regions participated in the audio-tactile binding, with both pleasant and unpleasant sounds. In addition, we observed a significant increase in beta-band power in medial visual areas, specifically when disagreeable movement sonification was paired with tactile exploration. This may reflect a disengagement of visual cortical processing, potentially amplifying auditory-driven emotional responses and intensifying the perceived unpleasantness of the explored surfaces.

Our results offer new insights into the neural mechanisms by which hedonic valence of auditory signals modulates emotional processing, without disrupting the perceptual analysis of texture properties.