Effects of Temporal Congruity Between Auditory and Visual Stimuli Using Rapid Audio-Visual Serial Presentation

Area of Science:

• Neuroscience
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Event-related potential (ERP)-based spellers increasingly integrate visual and auditory stimuli.
• Limited research addresses how temporal shifts in stimulus onset affect ERP components and system efficiency.

Purpose of the Study:

• To investigate the impact of temporal congruity between visual and auditory stimulus onsets in a bimodal brain-computer interface (BCI) speller.
• To analyze the influence of varying visual-to-auditory delays on ERP characteristics and classification accuracy.

Main Methods:

• Five bimodal paradigms with visual-to-auditory delays ranging from -33 ms to +100 ms were designed.
• Event-related potentials (ERPs) were recorded from 11 participants and analyzed using statistical methods on Fz, Cz, and PO7 channels.
• Classification accuracy was assessed based on visual-aligned ERPs to evaluate the effect of temporal delays.

Main Results:

• ERP component latencies were predominantly influenced by the visual stimulus onset.
• Auditory stimulus onset primarily affected early component accuracies, while visual stimulus onset determined later component accuracies.
• Visual stimulus onset played a dominant role in overall classification performance.

Conclusions:

• The temporal relationship between visual and auditory stimuli is critical for optimizing bimodal BCI speller performance.
• Understanding these temporal dynamics can guide the development of more efficient and effective BCI applications.