Speechreading enhancement by voice fundamental frequency: the effects of Fo contour distortions

Recognition of words in sentences of known topic was measured in normally hearing adults via speechreading alone and speechreading supplemented with auditory presentation of signals intended to convey variations of voice fundamental frequency (Fo) over time. Three signals were used: (a) the low-pass filtered output of an electroglottograph (unprocessed Fo), (b) a constant amplitude sine wave whose instantaneous frequency was intended to equal that of Fo (processed Fo), and (c) the same sine wave restricted to a small number of discrete frequency steps (quantized Fo). As the number of steps in the quantized Fo contours increased from 1 to 12, the speechreading enhancement effect increased. The quantized Fo contour with 12 steps was as effective as the processed Fo contour (without quantization), but this processed contour was significantly less effective than the unprocessed electroglottograph signal. The results show that the auditory Fo speechreading enhancement effect is sensitive to the errors introduced by the Fo extraction and regeneration process used in this study. It is also sensitive to the quantization of Fo contours into less than 12 steps. Whether more than 12 steps are required for the full enhancement effect remains to be determined.