Combining EEG, MIDI, and motion capture techniques for investigating musical performance

Area of Science:

• Neuroscience
• Music Cognition
• Motor Control

Background:

• Sequential recording of electrophysiological data (EEG), musical data (MIDI), and 3D movement provides valuable but separate insights.
• Music performance, a complex multisensory skill, benefits from integrated data capture.
• Previous methods lacked simultaneous recording capabilities for these data types.

Purpose of the Study:

• To describe a novel, low-cost setup for simultaneous recording of EEG, MIDI, and 3D movement data.
• To enable accurate temporal resolution for analyzing cognitive and motor processes during music performance.
• To facilitate behaviorally driven analysis of brain activity in music-related tasks.

Main Methods:

• Simultaneous synchronization of EEG and MIDI data using modified FTAP software with keypress event signals.
• Integration of a motion capture system to synchronize 3D movement data with recorded frames.
• Utilizing synchronization signals sent concurrently with data acquisition events.

Main Results:

• Successful simultaneous recording of electrophysiological, musical, and movement data.
• Accurate timing resolution achieved between the different data streams.
• A cost-effective solution for complex, multisensory performance analysis.

Conclusions:

• The described setup allows for simultaneous capture of brain activity, musical performance, and movement.
• This integrated approach offers a promising avenue for studying motor control, learning, and emotions in music.
• Enables more behaviorally informed analyses of neural processes during music performance.