Multi-modal cross-linguistic perception of fricatives in clear speech

Research shows that acoustic modifications in clearly enunciated fricative consonants (relative to the plain, conversational productions) facilitate auditory fricative perception, particularly for auditorily salient sibilant fricatives and for native perception. However, clear-speech effects on visual fricative perception have received less attention. A comparison of auditory and visual (facial) clear-fricative perception is particularly interesting since sibilant fricatives in English are more auditorily salient while non-sibilants are more visually salient. This study thus examines clear-speech effects on multi-modal perception of English sibilant and non-sibilant fricatives. Native English perceivers and non-native (Mandarin, Korean) perceivers with different fricative inventories in their native languages (L1s) identified clear and conversational fricative-vowel syllables in audio-only, visual-only, and audio-visual (AV) modes. The results reveal an overall positive clear-speech effect when visual information is involved. Considering the factor of AV saliency, clear speech benefits sibilants more in the auditory domain and non-sibilants more in the visual domain. With respect to language background, non-native (Mandarin and Korean) perceivers benefit from visual as well as auditory information, even for fricatives non-existent in their respective L1s, but the patterns of clear-speech gains are affected by the relative AV weighting and "nativeness" of the fricatives. These findings are discussed in terms of how saliency-enhancing and category-distinctive cues of speech sounds are adopted in AV perception to improve intelligibility.