Applying Deep Learning Techniques to Estimate Patterns of Musical Gesture.

Area of Science:

• Music Performance Analysis
• Motor Skill Learning
• Machine Learning in Music

Background:

• Repetitive practice is crucial for motor skill improvement in music.
• Violin bowing techniques require precise motor control and have distinct spatiotemporal dynamics.
• Objective feedback on technique accuracy is challenging to provide during practice.

Purpose of the Study:

• To analyze and classify eight traditional violin bow-strokes using forearm gesture data.
• To evaluate the effectiveness of deep learning models in recognizing these distinct bowing techniques.
• To explore the potential of these models for developing a digital practice assistant.

Main Methods:

• Recorded inertial motion data from experts and students using Myo sensors during violin bowing.
• Synchronized motion data with audio to capture spatiotemporal dynamics of eight bow-strokes.
• Applied and compared Convolutional Neural Network (CNN), 3DMultiHeaded_CNN, and CNN_LSTM deep learning models for gesture classification.

Main Results:

• CNN models achieved 97.147% accuracy, 3DMultiHeaded_CNN 98.553%, and CNN_LSTM 99.234% accuracy in classifying bowing techniques.
• The collected inertial data contained sufficient information to differentiate the studied bowing techniques.
• All investigated deep learning algorithms demonstrated high classification accuracies, confirming feasibility.

Conclusions:

• Deep learning models can effectively learn and distinguish patterns in violin bowing techniques from forearm motion data.
• The developed classifiers show promise for creating a digital assistant to provide real-time feedback on musical gestures.
• This technology could significantly enhance musicians' practice by offering objective insights into technique accuracy and consistency.