EEG-driven automatic generation of emotive music based on transformer

Area of Science:

• Computational Neuroscience
• Music Information Retrieval
• Artificial Intelligence

Background:

• Generating personalized emotional music is challenging due to difficulties in standardizing and mapping continuous electroencephalography (EEG) and audio data.
• Existing methods struggle with the complex, non-fixed relationship between EEG signals and musical elements like rhythm, melody, and emotion.

Purpose of the Study:

• To develop a novel method for emotional music composition using deep features from EEG data.
• To address the challenges of discrete representation and mapping between EEG signals and music data.

Main Methods:

• Utilized clustering algorithms to create discrete representations of EEG and music data, forming a standardized vocabulary.
• Employed a Transformer model with multi-head attention and positional encoding for reverse mapping of EEG signals to musical elements.
• Introduced an audio masking prediction loss for improved emotion-driven music generation.

Main Results:

• Achieved a Hits@20 score of 68.19%, a 4.9% improvement over existing methods.
• Demonstrated effective capture and utilization of temporal and structural relationships in heterogeneous data.
• Successfully enhanced the understanding of complex internal structures within EEG and audio data.

Conclusions:

• The proposed method offers a robust approach for personalized emotional music composition by effectively leveraging EEG data.
• The integration of clustering for discrete representation and Transformer models for mapping significantly improves music generation quality.
• This research highlights the potential of neuro-signal-driven music creation and its advancements.