Electroencephalogram Sonification with Hybrid Intelligent System Design Based on Deep Network

The electroencephalogram (EEG) sonification is an audio portrayal of EEG signals to provide a better understanding of events and brain activity thereupon. This portrayal can be applied to better diagnosis and treatment of some diseases.

In this study, a new method for EEG sonification is proposed based on extracting musical parameters and note sequences from the dominant frequency ratios and variations in the EEG signal. The ability of different classification structures in extracting musical scales and note sequences is evaluated. A music database has been created to train deep structures which, after extracting the frequency sequence of each piece of music as input, determines the scale label and note sequence in the output. A new algorithm is developed to combine the outputs of the deep structures and create a playable music repertoire.

The findings indicate that the convolutional neural network (CNN) classifier has an accuracy of 93.2% for the classification scales of musical pieces played in different octaves and 92.8% for pieces played in asymmetrical pieces. The convergence of EEG segments with musical scales is also reported for single channel, multi-channel of one person, different individuals, and different databases. The long short-term memory (LSTM) structure selected with an accuracy of 89.6% determines the sequence of notes.

The results show that the proposed CNN determines the appropriate and convergent musical scales with each EEG signal fragment and the LSTM network has a promising performance in converting the dominant frequency variations of EEG signals into note sequences. This demonstrates the good performance of the proposed sonification method.