Effect of impoverished information on multisensory integration in judgments of learning

According to the principle of inverse effectiveness (PIE), weaker responses to information in one modality (i.e., unisensory) benefit more from additional information in a second modality (i.e., multisensory; Meredith & Stein, 1986). We suggest that the PIE may also inform whether perceptual fluency affects judgments of learning (JOLs). If JOLs follow the PIE, the differences in JOLs for multisensory and unisensory items should increase as the unisensory study items become harder to perceive. That is, an influence of perceptual fluency should prompt a similar, interactive pattern across perceptual responses and JOLs. In 3 experiments, we systematically varied the signal intensity or noise in 1 modality to examine how responses might change with the inclusion of information in a second modality. In Experiment 1, written words in several font sizes were sometimes accompanied by spoken equivalents. In Experiments 2 and 3, spoken words in various background noise levels were sometimes accompanied by visual speech articulations. Consistent with the PIE, the multisensory benefits in response time and/or correct identification increased as responses to unisensory information decreased. Also, the multisensory formats received higher JOLs than the unisensory formats; however, unlike the predictions from PIE, this difference did not increase as study items became harder to perceive. Experiment 3 extended this finding to participants' explicit beliefs. In multisensory settings, JOLs may rely more on theory- than data-driven processes. We suggest that broadly defined processing fluency may always contribute to JOLs, but, regarding perceptual information, JOLs appear to track perceptual attributes rather than perceptual fluency. (PsycInfo Database Record (c) 2021 APA, all rights reserved).