Related Experiment Videos
Speech perception using combinations of auditory, visual, and tactile information.
P J Blamey1, R S Cowan, J I Alcantara
1Department of Otolaryngology, University of Melbourne, Royal Victorian Eye and Ear Hospital, Australia.
Journal of Rehabilitation Research and Development
|January 1, 1989
Summary
Adding visual lipreading significantly improved speech perception across all tested conditions. While auditory input also enhanced performance, tactile input showed limited effectiveness when combined with other senses, suggesting integration challenges.
Area of Science:
- Audiology
- Neuroscience
- Sensory Integration
Background:
- Speech perception relies on integrating multiple sensory inputs.
- Degraded auditory signals necessitate exploration of alternative or supplementary sensory channels.
- Multimodal sensory substitution holds potential for communication enhancement.
Purpose of the Study:
- To investigate the impact of combining auditory, visual (lipreading), and tactile (electrotactile speech processor) inputs on speech perception in normally-hearing individuals.
- To determine the effectiveness of each input modality and their combinations.
- To analyze information transmission and integration across senses.
Main Methods:
- Four normally-hearing subjects received training and testing with all combinations of auditory, visual, and tactile inputs.
- Speech perception was assessed using closed-set and open-set word/sentence recognition tasks, and speech tracking.
- Information transmission analysis was performed on vowel and consonant features.
Main Results:
- Visual input significantly improved speech perception in all conditions when added.
- Auditory input improved performance on most tests, except closed-set vowel recognition.
- Tactile input alone scored above chance but combined less effectively with other modalities, though it improved specific aspects like vowel and consonant recognition in certain combinations.
Conclusions:
- Visual input is highly effective in enhancing speech perception, even with degraded auditory signals.
- Auditory and visual information are integrated more efficiently than tactile information.
- Limited effectiveness of tactile input may stem from insufficient training or fundamental processing limitations in multimodal integration.