Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice.

Area of Science:

• Neuroscience
• Computer Science
• Biomedical Engineering

Background:

• Brain-computer interfaces (BCIs) utilize electroencephalography (EEG) signals.
• Multimodal stimuli, combining visual and auditory inputs, can enhance BCI performance.
• The effect of multimodal stimuli in rapid serial visual presentation (RSVP) BCIs remains unexplored.

Purpose of the Study:

• To propose and evaluate a rapid serial multimodal presentation (RSMP) BCI system.
• To investigate the impact of audiovisual stimuli on RSMP BCI performance.
• To identify key neural correlates, such as P300, contributing to classification accuracy.

Main Methods:

• Development of an RSMP BCI incorporating artificial facial images and artificial voice stimuli.
• Utilized scrambled images and masked sounds to isolate the effects of visual and auditory components.
• Employed EEG signal analysis to assess BCI performance and identify contributing neural signals.

Main Results:

• Audiovisual stimuli significantly improved the performance of the RSMP BCI.
• The P300 component, specifically at the Pz electrode, was identified as a key contributor to classification accuracy.
• The BCI system achieved a high online accuracy of 85.7 ± 11.5%.

Conclusions:

• The integration of audiovisual stimuli enhances RSMP BCI performance.
• The P300 signal plays a crucial role in the accuracy of this BCI paradigm.
• Findings support the development of advanced, gaze-independent BCI systems.