P300 audio-visual speller

Area of Science:

• Neuroscience
• Biomedical Engineering
• Rehabilitation Engineering

Background:

• The Farwell and Donchin matrix speller is a high-performing brain-computer interface (BCI).
• Its reliance on visual stimulation restricts use for individuals with visual impairments.
• Existing non-visual BCIs often exhibit lower performance or usability challenges.

Purpose of the Study:

• To develop a novel extension of the matrix speller BCI.
• To enable adaptable stimulation modalities (visual, auditory, simultaneous).
• To accommodate users with declining eyesight by allowing seamless modality transitions.

Main Methods:

• A modified matrix speller utilizing a 'flipping' letter matrix.
• Implementation of visual, auditory, and simultaneous visual-auditory stimulation paradigms.
• Testing with seven healthy subjects to evaluate performance across modalities.

Main Results:

• The modified visual + auditory stimulation slightly outperformed the classic matrix speller.
• The purely auditory system demonstrated lower performance than visual stimulation.
• Auditory BCI performance was comparable to existing auditory BCI systems.

Conclusions:

• The novel matrix speller extension effectively supports multiple stimulation modalities.
• This adaptable BCI design offers a promising solution for users with progressive visual loss.
• The system allows users to leverage optimal sensory input for their specific needs and conditions.