A Deep Learning Framework for EEG-Based Decoding of Visually Imagined Arrows with Different Colors and Directions

Area of Science:

• Neuroscience
• Computer Science
• Biomedical Engineering

Background:

• Brain-computer interface (BCI) systems offer potential in various fields.
• Visual imagery (VI) presents an alternative to motor imagery (MI) for BCI control.
• Existing VI-BCI paradigms have limitations in command complexity.

Purpose of the Study:

• To propose a novel BCI framework for classifying visually imagined arrows with combined color and direction attributes.
• To develop a new method for generating joint time-frequency-spatial representations (TFSR) from EEG signals.
• To evaluate the performance of a custom convolutional neural network (CNN) for decoding complex VI commands.

Main Methods:

• Utilized Choi-Williams time-frequency distribution (CW-TFD) to create TFSR from EEG data.
• Converted TFSR into grayscale images for input into a newly designed CNN.
• Collected a new EEG dataset from 16 subjects imagining 16 distinct arrow types (4 colors x 4 directions).

Main Results:

• Achieved an average classification accuracy of 95.05% and a Cohen's kappa score of 0.947 for 16-class decoding.
• The proposed CW-TFD-based TFSR and CNN framework outperformed alternative representations and classification models.
• Demonstrated consistent superior performance compared to pre-trained deep learning models and conventional machine learning classifiers.

Conclusions:

• The proposed framework successfully decodes an expanded set of visually imagined color-direction arrow commands.
• This study validates the feasibility of subject-specific EEG-based VI-BCI systems.
• Findings support the advancement of calibrated VI-BCI systems for assistive and interactive applications.