Virtual reality and voice measurements: Investigating auditory, visual, and audiovisual sensory streams

This project examines how speakers adjust their voices in response to multisensory and unisensory simulations in virtual reality.

Forty-one young adults produced reading and spontaneous speech samples in three simulation conditions: Multisensory audiovisual virtual reality, audio-only auralisations, and visual-only virtual reality environments. Acoustic voice parametres included sound pressure level, fundamental frequency, cepstral peak prominence smoothed, pitch strength, and time dose. Participants also provided self-reported ratings of vocal effort, fatigue, and discomfort on visual analog scales.

Multisensory simulations significantly influenced sound pressure level, fundamental frequency, pitch strength, and self-reported vocal effort compared to unisensory simulations. Female participants exhibited strong vocal adjustments to auditory input, while task demands (reading versus spontaneous speech) consistently altered vocal outcomes. Phonation time was elevated in audio-only auralisations.

Our findings demonstrate that voice production adjusts to audiovisual input in virtual reality, with multisensory simulations producing distinct outcomes from audio-only or visual-only conditions. Implementing multisensory simulations might improve voice recordings collected in traditional laboratory or clinical contexts, which may fail to reflect everyday communication. More broadly, the findings reinforce the multisensory nature of vocal motor control, as audiovisual input tended to exert stronger effects on voice outcomes than unisensory input.