Multimodal processing in simultaneous interpreting with text: Evidence from ear-eye-voice span and performance

Area of Science:

• Cognitive Psychology
• Linguistics
• Human-Computer Interaction

Background:

• Simultaneous interpreting (SI) with text is cognitively demanding due to combined auditory and visual input.
• Understanding how speech rate and experience affect multimodal processing in SI is crucial for interpreter training.

Purpose of the Study:

• To investigate the impact of source speech rate and professional experience on Ear-Eye-Voice Span (EIVS) and performance in Chinese-English SI with text.
• To analyze temporal coordination between auditory, visual, and verbal processes using eye-tracking.

Main Methods:

• 15 professional interpreters and 30 trainees performed SI with text tasks at varying speech rates.
• Eye-tracking technology measured Ear-Eye Span (EIS), Eye-Voice Span (IVS), and Ear-Voice Span (EVS).
• Interpreting performance was assessed across different speech rates.

Main Results:

• Faster speech rates reduced EIVS, indicating accelerated processing and adaptation.
• Professionals demonstrated more stable EIVS and better cross-modal attention, especially at high speech rates.
• Trainees showed greater visual reliance and struggled with multimodal demands, with higher EIVS variability.

Conclusions:

• Professional experience enhances multimodal coordination and cognitive resource management in SI with text.
• Faster speech rates challenge interpreters, but experience mitigates performance decline.
• Targeted training in auditory-visual coordination is vital for optimizing SI performance.