Electroencephalogram Sonification with Hybrid Intelligent System Design Based on Deep Network.

Area of Science:

• Neuroscience
• Music Technology
• Artificial Intelligence

Background:

• Electroencephalogram (EEG) sonification translates brain activity into auditory signals.
• This auditory portrayal aids in understanding brain events and can improve disease diagnosis and treatment.

Purpose of the Study:

• To propose a novel EEG sonification method.
• To evaluate deep learning classifiers for extracting musical parameters from EEG signals.
• To develop an algorithm for creating a playable music repertoire from EEG data.

Main Methods:

• EEG sonification based on extracting musical parameters and note sequences from dominant frequency ratios and variations.
• Training deep learning structures (CNN, LSTM) using a music database to identify musical scales and note sequences.
• Developing a novel algorithm to combine deep structure outputs for music generation.

Main Results:

• Convolutional Neural Network (CNN) achieved 93.2% accuracy in classifying musical scales and 92.8% for asymmetrical pieces.
• Long Short-Term Memory (LSTM) achieved 89.6% accuracy in determining note sequences.
• Demonstrated convergence of EEG segments with musical scales across various data types.

Conclusions:

• The proposed CNN effectively identifies musical scales corresponding to EEG signal fragments.
• The LSTM network shows promise in converting EEG frequency variations into note sequences.
• The EEG sonification method demonstrates good performance in translating brain activity into music.