Audiovisual Speech Perception With Less Familiar and Frequent Words

Lexical factors, such as word frequency and neighborhood density, impact word recognition accuracy in challenging listening environments, within both auditory and audiovisual (AV) conditions. Here, we investigate how an understudied lexical factor-word familiarity-impacts word recognition in both audio (A)-only and AV conditions using an ecologically valid context of medically related sentences embedded in a simulated hospital soundscape.

Monolingual adult listeners between the ages of 18 and 35 years (N = 98) were presented with 160 sentences mixed with a simulated hospital noise in A-only and AV conditions. The stimuli included both medically related sentences with varied lexical frequency and lexical familiarity profiles and non-medically related sentences. The listeners typed what they heard after each sentence. Responses were scored based on key word accuracy. Participants also completed a background questionnaire.

Word recognition was higher in AV than A-only condition regardless of lexical characteristics. Word recognition was also more accurate for words with higher lexical frequency and familiarity. An interaction between frequency and presentation modality suggests that the AV benefit is larger for low-frequency words. Neither listeners' self-rated knowledge of medical terminology nor experience in medical settings related to word recognition accuracy.

Visual speech cues support word recognition even for words that are low in familiarity for most listeners. Thus, when health care providers convey diagnosis and treatment information to patients that includes less frequent or familiar terminology, face-to-face communication may be particularly important.