Blurring of Visual Images Facilitates the Effect of Semantically Congruent Sounds on Visual Memory Performance

Our perceptual/cognitive mechanisms effectively use multisensory information to recognize the external world. In the memory domain, visual memory performance is reported to be improved when target images are presented with semantically congruent sounds. Multisensory interactions are demonstrated to be more evident when less clear/effective sensory inputs are presented (the principle of inverse effectiveness). It remained unclear whether audiovisual interactions in visual memory performance could differ depending on the visibility of the stimuli. The current study examined whether the visibility of target visual stimuli could modulate the enhancement effect of semantically congruent sounds on visual memory performance. In an encoding session, we asked participants to memorize pictures presented with semantically congruent or meaningless sounds. In the following recognition session, participants were asked to judge whether a series of pictures, which were always unblurred, had been presented in the previous session without hearing any sounds. The first experiment showed that the semantically congruent sounds did not enhance visual memory performance when unblurred pictures were presented during the encoding session. In the second experiment, we presented blurred pictures during the encoding session and found better recognition performance for the pictures with the semantically congruent sounds. The third experiment revealed no enhancement effect when the semantic congruencies were violated between the sounds and blurred visual images during the encoding session, suggesting that response or decisional biases were not critical factors. Our findings suggest that our perceptual/cognitive mechanisms utilize audiovisual information for visual memory, especially when available visual information is less effective.